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Abstract 


This  research  augments  formed  languages  with  the  machinery  necessary  to  describe 
labelled  combinatorial  objects  such  as  trees,  permutations,  and  networks.  These  objects 
typically  have  requirements  for  their  labels  (trees,  for  example,  can  be  (Hinivalent  under 
permutation  of  subtrees)  that  make  certain  labellings  invalid  or  redundant.  To  deal  with 
this  problem,  formal  languages  cire  augmented  with  partial  orders— derived  strings  have 
partial  orders  specifying  acceptable  labellings,  and  productions  of  the  grammar  contain 
fragments  of  partial  orders.  The  traditional  rewrite  stop  in  a  derivation  is  now  coupled 
with  a  substitution  that  joins  two  partial  orders. 

The  most  attractive  feature  of  this  method  of  describing  combinatorial  objects  is  the 
direct  translation  to  generating  functions.  Treating  the  grammar  of  an  ordiimry  formal 
language  as  a  set  of  equations  and  then  solving  these  equations  yields  an  enumerating 
generating  function.  This  is  still  true  of  labelled  formal  languages  although  the  equations 
arc  usually  differential  rather  than  rational  or  algebraic. 

There  are  two  promising  applications  for  labelled  formal  languages.  In  the  analysis 
of  algorithms  one  often  identifies  combinatorial  quantities  that  can  be  described  with  la¬ 
belled  formal  languages  and,  using  the  translation  mentioned  above,  these  quantities  can  be 
easily  computed.  The  other  €app]icatioTi  uses  labelled  formal  languages  to  control  a  general- 
purpose  system  for  the  ranking,  sequencing,  and  selection  of  combinatorial  objects.  Both 
of  these  applications  demonstrate  the  value  of  labelled  formal  languages  as  a  descriptive 
and  analytic  tool. 
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Chapter  1 


Introduction 

The  remarkably  simple  concept  of  a  grammar  has  led  to  considerable  clarity  in  the 
development  of  computer  science. 

Definition.  A  grammar  ;s  a  four-tuple  (T,NyS,P)  consisting  of 

1)  a  terminal  alphabet  T  (usually  small  letters); 

2)  a  nonterminal  alphabet  N  (usually  capita/  letters}; 

5}  a  start  symbol  S  G  N; 

4)  a  set  of  productions  P. 

The  heart  of  a  grammar  is  item  4,  the  set  of  productions  that  gmde  the  generation  of 
strings. 

Definition.  A  production  is  a  rewrite  rule  with  two  strings  over  the  alphabet  TUN. 
The  left  hand  string  is  separated  from  the  right  hand  string  by  an  arrow,  for  example, 
AD  — ►  cDeF, 

Definition.  A  derivation  step  consists  of  matching  a  substring  with  the  left  band 
string  of  a  production  and  then  rewriting  it  with  the  right  hand  string. 

Definition.  A  string  is  derived  by  a  grammar  if  it  is  possible  to  obtain  the  string 
from  the  start  symbol  of  the  grammar  with  a  finite  number  of  derivation  steps. 

In  the  remainder  of  this  work,  grammars  will  always  be  context-free,  that  is,  their 
productions  will  always  have  a  single  nonterminal  on  the  left  hand  side.  We  will  usually 
call  these  “grammars”  even  though  they  arc  more  properly  CcUled  “context-free  grammars” 
in  the  study  of  formal  languages.  It  is  oflen  convenient  to  group  several  productions  with 
a  common  left  hand  side  using  vcrticcd.bars  to  separate  the  production  possibilities.  So, 
for  example,  A  B\c\d  will  denote  three  productions  A  P,  A  C,  and  A D. 
The  greek  c  is  used  as  a  symbol  for  the  null  string. 
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INTUODtiCTION 


Grammar?  are  used  primarily  for  description;  they  specify  a  set  of  acceptable  strings, 
called  a  formal  language.  For  example,  if  we  are  encoding  trees  by  their  walks,  with  I  for 
lower,  h  for  higher,  and  x  for  a  tree  node  as  shown  in  Figure  1.1,  then  the  grammar 

T^xF 

F  -*  IThF  I  c 

specifies  the  language  of  valid  trees.  The  first  production  means  that  every  tree  is  a  root 
node  connected  to  a  forest  of  offspring,  while  the  second  production  defines  a  forest  as  a 
sequence  of  trees.  Starting  with  the  symbol  T,  the  tree  of  Figure  1.1  is  derived  as  follows: 

T 

xF 

xlThF 

xlxFhF 

xlxhF  (1.2) 

xlxhlThF 

xlxhlxFhF 

xlxhlxhF 

xixhlxh . 

Considerable  research  has  beeji  devoted  to  the  inversion  of  this  process  (parsing  a  string  to 
obtain  a  derivation)  and  to  the  study  of  the  descriptive  power  of  formal  languages. 

The  focus  of  this  dissertation  is  on  another  interesting  property  of  formal  languages. 
If  the  productions  of  a  grammar  arc  treated  ius  equations,  with  -»  replaced  by  = ,  |  by  + , 
and  i  by  1,  mid  if  the  ciiuatioius  are  solved  for  the  stmt  symbol,  the  result  is  a  generating 
fimetion  for  the  number  of  derivations  of  the  grammm. 

Generating  functions  have  become  a  central  tool  of  combinatorial  mathematics;  they 
me  used  to  study  sequences  of  numbers  {£7,}<>o.  The  <;,■  me  implanted  as  the  Taylor 
coefficients  of  a  function  G  of  a  dummy  variable  x: 


(1.3) 
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ai)d  then  the  properties  of  the  seqoenee  are  studied  through  the  properties  of  G{z).  Gen¬ 
erating  funetions  have  various  flavors.  We  will  be  u.sing  ordinary  generating  functions  like 
(1.3)  in  this  ehapter,  and  exponential  generating  funetions  in  the  following  chapters.  An 
exi)onential  generating  function  has  the  form 

=  (1-4) 


In  our  applications,  the  3.  coefficients  will  count  the  number  of  derivations  of  a  grammar. 
Grammar  (1.1)  converts  to  two  equations: 


r  =  iF 
F^lThF  +  l. 


(1.5) 


We  solve  for  T  by  algebraically  eliminating  all  other  nonterminal  characters  and  by  treating 
terminal  characters  as  dummy  variables. 


F  - - 

1  -  IhT 

T  -  IhT^  =  I  (1.6) 

1  ±  ^/T~-~4^hx 

T  = -  .  -  . 

2lh 

The  extraneotis  root  is  discarded  with  the  observation  that  the  grammar  cannot  derive  an 
empty  tree,  and  so  r(0)  must  be  zero.  The  remaining  root  is  expanded: 


1  -  y/l  -  Alhx 
2lh 


(_1)>-«2*>"‘1>‘‘/»^"V 


(1.7) 


We  obtain  a  multivariate  generating  function  where  the  coefficient  of  is  the  number 

of  strings  having  a  /’a,  b  h\  and  c  x’s.  However,  I  and  h  are  not  particularly  useful;  there 
will  always  be  one  less  of  these  WcJk  control  characters  than  there  arc  tree  nodes.  Hereafter 
wo  shall  drop  such  irrelevant  characters  by  replacing  them  with  1  cfirlier  in  the  analysis.  We 
conclude  that  the  number  of  trees  with  j  nodes  is  espial  to  the  Catalan  number  j  • 

This  close  conni'ction  between  formal  languages  and  power  series  appe.ared  in  the  early 
work  of  Schiitzcnbcrgcr  and  Chomsky  [Schiitzen  1901]  [Choniskv  1903].  They  found  that 
power  series  over  noncommutativc  varitibles  could  be  very  helpful  in  the  classification  of 
formal  languages:  regular  expressions  became  rational  equations,  and  context  free  languages 
became  algcbrai'*  equations.  Issues  of  ambiguity  were  now  questions  about  the  coefficients 
of  power  series.  A  full  cliscussion  of  this  line  of  rcsciirch  can  be  found  in  the  recent  work  of 
Salomaa  and  Soittola  [Salomaa  1078]. 
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INTTU)nurTI()N 


Nfaurirp  (»ross  was  rrsptvnsiMr  for  fakinj;  tins  liiu*  of  rrsi'an  h  in  tlu*  tlirrrtion  of  rnii* 
incration  of  coinl'inatorial  objrrts  [(Jross  lOGGj.  Ho  roaliz<'(l  that  hy  coininntinj;  the  variabh’S 
of  the  power  sc'ries,  one  rouhi  eo}Ie<‘t  in  t!ir  roc'fiirients  all  words  with  tlie  same  composi¬ 
tion  of  terminal  ( iiaracters.  (In  the  examph'  al)ove.  we  collected  tojr^'ther  all  tiers  with  the 
same  ninnher  of  nodes.)  This  meant  that  if  a  combinatorial  objtrt  could  bo  described  with 
a  string;  of  charactiTs,  and  if  the  valid  striiij^s  cotdd  be  dc'scribi'd  with  a  grammar^  then 
it  was  easy  to  obtain  a  j^eneratinj’  function  for  the  family  f>f  combinatorial  objects.  The 
grammar  ha<l  to  he  relatively  simple  (regular,  or  context  fnv)  and  had  to  derive  uriainbigu- 
onsly  the  language  of  valid  strings;  nev<TtheIess.  this  tcnlinique  provided  straightforward 
enunieratitms  fi>r  several  complex  combinalorical  families. 

The  aim  of  this  dissertation  is  to  extend  the  aj)j)licahihty  of  form;J  languages  to  a 
wide  range  of  lahelh'd  eomhinatorial  objects.  This  extension  is  developed  in  Chapter  2. 
Chajiters  3  and  4  explore'  the  «advantagcs  of  the  extension,  using  it  to  analyze  algorithms 
and  to  generate  combinatorial  ohje^cts.  But  lirst  of  all.  the  rem.ainder  of  this  chapter  is 
devoted  to  a  survey  of  tlie  enumerative  uses  of  formal  langmages.  in  order  to  describe 
techniejues  that  will  be  helpful  in  later  chapters  and  to  understand  (he  limitations  of  formal 
languages. 

1.1  Variations  of  Trees. 


In  a  fundamental  sense,  every  grammar  describes  a  family  of  trees.  (These  are  the 
deriv«ation  trees  for  the  valid  strings  of  the  language.)  So  it  is  not  surprising  tlnat  context  free 
languages  prove  useful  in  the  enumeration  of  objects  having  an  underlying  tree  structure, 
oven  though  on  the  surface  th(‘  jiroblem  may  be  unrelated  to  trees.  Maurice  (^ross  gives  the 
folIi)wing  example:  A  ternary  triangulation  is  formed  by  repeatedly  dividing  single  triangles 
into  three  parts,  where  the  division  is  accomplished  by  adding  a  new  vertex  in  the  interior 
of  the  triangli'  and  connecting  it  to  the  three  corners.  A  sample  triangnlation  is  shown  in 
Figure  1.2. 


Figure  1.2  A  Ternary  Triangulation 

Triangnlations  arc  encoded  with  parentheses:  EJach  triangle  is  represented  by  a  bal¬ 
anced  pair  of  parentheses,  (  ),  and  when  a  triangle  is  subdivided  three  new  pairs  arc 
introduced  inside  the  original,  (()()()),  representing  the  new  triangles  in  clockwise  order, 
starting  with  the  triangle  on  the  base  of  the  enclosing  trhuiglc.  The  biisc  etlgcs  of  the 
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new  triangles  arc  those  o<lges  in  common  with  the  sides  of  the  old  triangle.  Following  this 
procedure,  the  triangulation  pictured  in  Figure  1.2  is  encoded  os 

(Olcoooioo)!))- 


The  grammar  for  this  language  guarantees  that  the  interior  of  each  pair  of  parentheses 
will  bo  divided  into  three  parts  or  none  at  all: 

S-^(555)|().  (1.8) 

We  would  like  to  count  triangulations  according  to  the  number  of  triangles,  so  is  mapped 
to  X,  and  is  mapped  to  1: 

5  =  xS*-bx.  (1.9) 

Rather  than  solve  this  cubic  equation  for  S,  there  is  an  easier  technique,  due  to  Lagrange, 
that  allows  us  to  find  the  coefficients  of  S  without  knowing  a  closed  form  for  S  itself.  In 
general,  if  we  have  a  functional  equation  for  5, 

S  =  x/(S),  (1.10) 


and  if  we  let  (x”)S  denote  the  coefficient  of  in  the  Taylor  series  for  5,  then  Lagrange’s 
inversion  formula  gives  us: 

(i")S  = -(S"-*)/(5)".  (1.11) 

n 

Further  discussion  of  this  formula  can  be  found  in  Appendix  A  (which  is  oriented  towards 
exponential  generating  functions  as  opposed  to  the  ordinary  generating  functions  of  this 
chapter),  and  in  the  next  example  of  this  section.  For  the  current  example  we  obtain: 


(i")5=  i(S"-*)(l  +  S’)" 


n  =  3j  +  1 
otherwise. 


(1.12) 


For  fiirthcr  enumeration  problems  with  underlying  tree  structure  see  the  works  of 
Maurice  Gross  [Gross  1966],  Jay  Goldman  [Goldman  1978  and  1979),  W.  Kuich  [Kuich 
1970a  and  1970b],  and  Vaughan  Pratt  [Knuth  1973;  problem  2.2.1“ll|.  Several  other  prob¬ 
lems,  although  not  analyzed  originally  with  foriinil  languages,  also  exliibit  tree  structure: 
SchrmltT’s  second  problem,  Fibonacci  sequences,  and  occupancy  pn)blenis  (with  unlabcllcd 
balls  placed  in  an  ordered  collection  of  boxes)  ciui  all  be  encoded  with  context  free  lan¬ 
guages. 
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INTRODUCTION 


The  pervasive  tree  structure  suggests  that  we  analyze  the  most  general  tree  possible, 
cast  in  the  form  of  functional  composition  [Goldman  1070]: 

r-Fo|c,  |Fa|  ... 

Fo  -*  /oi  I  /cJ  I  •  •  •  j  /Oe, 

F,-/n(T)l/n(r)|/,3(r)|  ...  |/,.,(2') 

Fa  -  /ai (r, T)  \  MT, T)  |  . . .  |  {T, T) 

Fj-»/3,(r,r,r)|/3a(r,r,r)| ...  |/j.,(r,r,r) 


A  term,  T,  is  cither  a  constant  like  /oi,  or  a  function  applied  to  several  terms  such  as 
/2i(r,  r).  The  first  index  of  an  /  symbol  indicates  the  number  of  operands  expected  by 
the  function.  In  order  to  count  all  possible  compositions  according  to  the  number  of  terms 
present,  we  map  the  /'s  onto  x.  This  Iccaves  an  implicit  equation 

r  =  iC(r)  (1.14) 

where  C(T)  has  coefficients  equal  to  the  number  of  functions  of  each  degree: 

C(T)  =  ^ejTi.  (1.15) 

3>0 

Lagrange’s  inversion  formula  is  well  adapted  to  invert  an  equation  like  (1*14): 

,U6, 

But  here  Raney  has  develope<l  the  following,  altenmte,  approach  to  the  problem  that 
gives  an  interesting  constructive  interpretation  of  Lagrange's  formula  [Raney  1900).  First, 
notice  that  no  information  is  lost  by  dropping  the  parentheses  from  a  composition  of  func¬ 
tions, 

/lo(/2l(/oi./o2))  ,, 

/..A./../,., 

because  the  functions  appear  in  an  unambiguous  Polish,  or  prefix  order.  However,  not 
every  list  of  functions  is  a  valid  composition;  we  cannot  always  add  parentheses  and  obtain 
a  single  term. 

When  is  a  list  of  functions  a  valid  composition?  The  answer  uses  the  following  scheme 
of  Lukasiewicz:  Functions  requiring  j  operands  are  given  a  weight  of  j  -  1,  so  the  total 
weight  of  a  complete  term  will  always  be  -1.  Continuing  the  example  above,  /10/21/01/02 
has  weights  of  0,  1,  -1,  and  —1  for  a  total  of  —1,  as  expected. 

Suppose  that  the  we  pimble  together  a  collection  of  /'s  with  a  total  weight  of  —1,  is 

this  neces.sarily  a  valid  composition?  Not  always,  but  curiously  one  (and  only  one)  of  the 

cyclic  pennutations  of  our  list  will  be  a  valid  composition.  The  reason  lies  in  an  additional 
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constri^int  on  tlie  weights.  Not  only  must  they  sum  to  —1,  but  a  cumulative,  left  to  right 
sum  of  the  weights  must  remain  greater  than  —1  until  the  last  /.  This  avoids  a  premature 
completion  of  the  term. 

If  we  graph  the  sum  of  the  weights  of  the  first  j  terms  as  a  fiinction  of  j,  then  a  typical 
composition  looks  something  like  Figure  i.3  below. 


Figure  1.3  The  Cumulative  Weights  of  a  Functional  Composition 

Notice  that  any  cyclic  permutation  of  these  letters  cannot  be  a  valid  composition  since 
point  A  would  have  a  y  component  less  than  or  equal  to  -1,  and  would  appear  before  the 
last  position  in  the  permutation.  Conversely,  if  we  take  any  collection  of  /s  summing  to 
-1,  find  the  point  with  smallest  y  component  (breaking  tics  in  favor  of  the  leftmost  point), 
and  cyclically  shift  this  point  to  the  last  position,  then  the  new  graph  will  remain  positive 
until  the  end,  indicating  a  valid  composition. 

To  derive  Lagrange’s  inversion  formula  we  simply  interpret  the  Last  few  paragraphs  with 
generating  functions.  Starting  with  C{r),  which  has  coefficients  Cj  equal  to  the  number 
of  functions  taking  j  oporcands,  we  can  weight  the  functiu!is  according  to  Lukasiewicz  by 
dividing  by  T:  C[T)IT. 

Next  we  select  N  functions  with  total  weight  —1, 


and  we  know  that  only  one  cyclic  permutation  will  be  correct,  so  there  is  a  1/iV  chance 
that  we  have  a  valid  compositional  pattern: 

.  (1.19) 

This  is  Lagrange’s  formula  for  its  inversion  of  T  =  xC(T). 

Tlic  functioned  composition  problem  explains  why  Lagrange’s  formula  is  so  often  seen 
in  the  solution  of  grammar  related  tHpiations.  By  adjusting  c,-,  the  number  of  functions  with 
I  operands,  we  arc  controlling  the  branching  of  a  tree.  M<uiy  problems  arc  special  eases 
of  the  last  analysis.  For  example  the  triangulatiou  griunimir  appciuing  at  the  beginning 
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of  tins  section  Uranclu's  tliree  ways  (03  —  1)  and  tliere  is  only  one  way  to  tenninato  the 
branching,  (co  =  1).  The  remaining  nx'flicients  are  zero.  This  means  that  our  earlier 
aj)pli( <ition  c)f  Lagrange  s  inversioTi  formula  in  ccpiation  (1.12)  is  r<]uivalent  to  treating 
functional  composition  of  a  ternary  function  and  a  constant. 

The  generality  of  the  last  restilt  also  points  to  a  limitation  in  the  use  of  grammars  in 
eininieration.  The  probh'ms  imiy  be  complicated  by  several  nonterminals  (corresponding 
to  constraints  on  the  composition  of  functions)  but  it  is  hard  to  get  beyond  the  basic  tree¬ 
like  nature  of  these  gramnnvrs.  Nevertheless,  some  extensions  are  possible.  The  next  few 
sections  are  devoted  to  techniques  tliat  increase  the  versatility  of  context  free  languages. 


1,2  Indexing  and  the  Q  Operator 
Consider  the  following  problem: 

Problem.  Count  the  number  of  partitions  of  n  into  seqfiences  of  positive  ititcgcTS 
U  <  *2  <  *3  <  •  ‘  ■  <  im  that  t .  + 1*2  +  *  •  *  +  im  =  n,  and  the  parts  of  the  partition 
arc  bounded  both  in  number  (m  <  j)  and  in  size  (im  <  k). 

To  solve  this  problem,  partitions  are  encoded  with  blocks  off/’s  separated  by  x's  and  Cs. 
The  gs  encode  the  parts  of  the  partition,  the  x^s  mark  the  end  of  each  block,  and  the  Cs 
signal  transitions  in  block  size.  Using  this  system,  a  partition  of  g  into  1  -p  1  +  3  +  4  would 
appear  as 

tgxgxttgggxtqgggx .  (1.20) 

However,  it  is  impossible  to  express  a  string  like  (1.20)  with  a  finite  grammar,  since  the 
grammar  could  not  insure  that  block  sizes  iiirrefa.se.  This  difficulty  is  remedied  with  a 
countably  infinite  number  of  nonterminals.  The  grammar  accounts  for  block  size  in  the 
index  of  the  nonterminals: 


je  t>0.  (1.21) 

If  wp  start  with  Sq,  the  grammar  has  two  quirks;  it  can  gciicratp  an  arbitrarily  long  string 
of  I's  before  it  i)rodures  any  blocks  of  q'»-,  and  it  can  app.-nd  an  arbitrarily  long  string  of 
t  s  to  the  end  of  a  p.ortifion.  Both  of  these  quirks  are  helpful  in  solving  the  problem  as 
originally  poseil.  If  wo  count  all  strings  with  j  x’s,  then  we  will  enumerate  partitions  with 
j  or  fcw<'r  i)arts.  Likewise,  if  wo  examine  strings  with  k  t's  then  the  niaxiinum  part  size 
will  be  less  than  or  equal  to  k. 

The  remaining  challenge  is  to  solve  an  infinite  set  of  equations  for  5o: 

5<  =9‘iS<  +  15,>, +  1  j>0.  (1.22) 

This  is  accomplishc<l  by  applying  the  Q  t.p.Tator,  Qf{x)  =  f[qx),  to  equation  (1.22)  and 
noticing  that  wo  obtain  the  equation  for 

QSi  =  q*^^xQSi  +  tQ 5,+i  +  1  »  >  0 .  (1.23) 

So  we  expect  that  QSi  —  S, , ,  in  which  case  the  infinite  set  of  equations  collapses  to  a 
single  equation  for  .Vq: 


^0  —  xSq  +  1  Q  Sq  +  1 . 


(1.24) 


INDEXING  AND  THE  Q  OPERATOTl  0 


More  formally,  we  have  the  following  theore.  j: 

Fixed  Point  Theorem.  Let  Si  be  given  by  any  syste:.i  of  equations  of  the  form 

Si  =  q'0i{Si,Si^i)  +  02{Si,Si^i)  i>0.  (i.25) 

with  oper/itors  Oi  mid  O2  such  that 

QOi{A,n)  =  qOt(QA,QB)  and  Q02{A,  D}  =  02{QA,QB) .  (1.26) 

Let  S*  be  a  solution  of 

S*  =  Oi(S\QS‘)  +  03(5*,Q5*)  ,  (1.27) 

then  Si  =  Q'S*  will  satisfy  the  system  (1.25). 

Proof.  Apply  Q*  to  (1.27). 

The  solution  of  (1.24)  follows  the  pattern  of  binomial  cocfficien's.  We  know  that, 
without  the  Q  operator  (i.c.,  when  q  =  1), 


So  ~  "h  ^^0  "h  1 

1 


So  = 


1-i-t 


.*  \  J  / 


J.Jk>0 

We  claim  that  with  the  Q  operator: 

So  =  xSo  d-tQ  So  I 

l-x-iQ 


so=  E  C';*) 


(1.28) 


wlicrc  the  following  definitions  make  the  analogy  work: 

j!,  =  (l-9)(l-7’)...(l-mi-«)» 
(l-q)(l-q»)...(l-q^>*) 


(j  +  k\  (l-q)(l-q^}...(l-q^n 
\  j  A"  (l-«7)  ..(l-7^)(l-9). .  (1-9^) 

m>0 


(1.20) 


(1.30) 


We  assume,  in  the  last  sum,  that  the  Q  operators  are  applied  to  all  factors  on  their  right. 
Further  properties  of  the  Q  operator  and  g-nomial  coeflBcicnts  cim  be  found  in  [Andrews 

mij. 
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Efiiiatioiis  witli  tlio  Q  operator  arc  not  always  ;vs  clean  as  the  above  example.  Our  next 
example  uso.s  the  chain  rule  to  extract  usehii  information  from  a  more  ditiicult  sitiietion. 
It  is  a  technique  that  ai)peare<l  originally  in  the  solution  of  problem  2  3.4.5  5  in  [Kniith 
1973],  and  will  prove  tiseful  later  in  the  analysis  of  algorithms. 


Problem.  WhM  i.s  the  average  intcrnnl  path  length  of  an  {utlahellcd  tree?  (Internal 
path  length  is  the  sum  of  the  distances  from  each  tree  node  to  the  root.) 


By  modifying  the  grammar  found  at  the  beginning  of  this  chapter  we  can  preface  each 
node  of  the  tree  with  a  block  of  q's.  The  length  of  the  block  of  q's  is  equal  to  the  depth  of 
the  node  within  the  tree: 

Ti-^q'xFi 


Fi-*lTi,thFi\e. 


(1.31) 


An  application  of  the  Q  operator  reveals  that  QTi  =  Tj+i  so  once  again  the  infinite  set  of 
equations  can  be  reduced  to 


To  =  xFo 

^0  =  (QTq)Fq  -f  1 . 


(1.32) 


This  time  the  solution  is  not  closed: 


To 


X 

rr^- 


(1.33) 


It  can  be  expanded  into  a  well  known  continued  fraction  of  Ramantyan, 


ro  = 


1  - 


qx 


1  - 


q^x 


1- 


q^x 


(1.34) 


but  for  our  purposes  we  do  not  need  a  complete  solution,  only  an  average  path  length.  This 
suggests  dilfercntiating  expiation  (1.33)  with  respect  to  q  and  setting  q  equal  to  1.  Let 


We  have  already  computed 


(1.35) 


P(x)  =  r„|,.,=  (1.30) 

Rewriting  equation  (1.33)  and  then  differentiating  the  equation  gives 


To  —  TqQTq  =  X 
U -UV  -V{V'x  +  U)  =  0, 


(1.37) 
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where  the  tcnii  in  parentheses  is  the  chain  mlc  applie<l  to  QTq.  Algebraic  manipulation 
yields: 

xVV 

U  = 


1-2V 


.,x  1- vT^ 
2  1  -  4i 


(1.38) 


For  an  average  path  length  we  divide  by  the  total  number  of  trees: 


n 


(1.30) 


1*3  Differencing 

The  next  example,  taken  from  [Gross  19CC],  illustrates  another  technique  for  dealing 
with  languages  that  do  not  have  grammars  in  the  usual  sense.  The  problem  is  to  enumerate 
Tutte  triangulations. 

DeBnition*  A  Tiitte  triangiilation  is  a  division  of  a  polygon  into  triangles  such  that 
no  internal  edges  have  both  their  endpoints  on  the  boundary  of  the  polygon.  One  of 
the  externa]  ed^cs  is  niarlccd  with  an  orientation,  as  shown  in  Figure  1,4  below. 


Figure  1.4  The  Tutte  IViangulation 

A  triangulation  is  assembled  from  in<lividu<al  triangles  that  are  dcnotc<l  by  t’s  in  the  for¬ 
mal  Language.  Initially  each  triangle  has  a  nuuked,  counterclockwise  oriented  edge.  The 
structure  is  aggregated  by  two  operations.  The  fanning  operation,  /,  joins  two  structures 
by  abutting  the  edge  following  the  marked  c<lge  of  the  first  structure  with  the  marked  edge 
of  the  second  stnicture.  The  expression  tfi  is  shown  in  Figure  1.5  below.  Notice  that  the 
marked  edge  of  the  first  structure  remains  the  marked  edge  of  the  whole  structure. 
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Figure  1.5  The  Fanning  Operation 

The  closing  operation,  c,  creates  a  new  marked  edge  that  is  connected  to  the  tail  of  the  old 
marked  edge  and  the  far  end  of  the  edge  following  the  marked  edge.  The  closure  of  Figure 
1.5,  c[tft]^  is  pictured  in  Figure  1.6. 


Figure  1.6  The  Closing  Operation 

The  grammar  for  Tutte  triangulations  specifics  that  the  outermost,  or  last  operation 
must  be  a  closure,  to  prevent  internal  edges  from  bisecting  the  triangulation: 


S  c(i2]  I 

^S\RfS 


If  we  strip  off  the  outermost  closure,  what  remains  is  a  triangiilatiou  that  allows  bisecting 
edges  to  touch  the  head  of  the  marked  edge.  These  arc  represented  by  R  in  the  above 
grammar;  R  can  be  decomposed  by  notmg  that  R  is  either  itself  a  Tutte  triangulation, 
with  no  bisecting  edges,  or  we  can  find  the  rightmost  bis(K:ting  edge  and  split  off  a  Tutte 
triangulation. 

However,  there  is  a  serious  flaw  in  (1.40),  It  makes  no  sense  to  close  a  single  triangle, 
and  in  genered,  if  a  structure  has  j  external  edges  wc  may  apply  c  only  y  -  3  times  before 
an  external  triangle  results.  In  order  to  avoid  closing  a  triangle  we  would  like  to  constrain 
the  c’s  to  be  strictly  less  than  the  enclosed  fs,  yet  this  does  not  seem  possible  with  a 
context-free  grammar. 

The  following  differencing  trick  allows  us  to  derive  a  correct  equation  from  the  grammar 
at  (1.40).  Lot  T  be  all  strings  representing  Tutte  triangulations  where  the  exterior  polygon 
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is  a  triangle.  Since  S  represents  till  'I\itte  triangulations,  T  is  a  subset  of  S,  containing 
strings  wi'  h  one  more  t  than  c.  If  a  string  derived  with  (1.40)  violates  the  constraint  on  c  s 
and  t's,  then  somewhere  within  the  string  a  structure  with  triangular  exterior  is  closed  by 
a  c  operation.  We  prevent  this  from  happening  by  subtracting  the  language  cfT)  from  the 
first  nonterminal; 

S-«|fl||(  -c|Tl 

R^s\RfS. 

Subtraction  means  that  we  eliminate  a  set  of  strings  from  the  language  derived  from  a 
nonterminal.  If  the  strings  eliminated  are  a  proper  subset,  as  is  the  case  here,  we  can 
proceed  with  a  translation; 

S  =  cR  +  t-cT 

f  (1.42) 


R  = 


1-/5- 


These  equations  combine  to  give; 


S  +  cT  =  t  + 


cS 


l-fS' 


(1.43) 


The  will  always  be  one  more  t  than  /,  so  t  can  be  replaced  by  1.  Now  T(/,c)  is  simple 
the  diagonal  terms  of  S{/,c),  and  most  features  of  interest  are  related  to  the  occurrences 
of  /  and  c,  denoted  respectively  by  7  and  C: 

Interior  Triangles  =  7  -h  C  +  1 
Exterior  Edges  =  7  -  C  +  3 
Interior  Edges  =  7  +  2C . 

Using  a  sophisticated  application  of  Lagrange’s  theorem  to  equation  (1.43),  Tutte  was 
cable  to  find  an  expansion  for  S  [Tutte  10G2;  pages  26~*31].  The  number  of  triangulations 
according  to  7  and  C  turns  out  to  be 


for  7  =  C,  and 


2(4C  +  1)! 

(3C  +  2)!(C  +  1)! 


(1.45) 


3(7  _  C  +  2)!  {T-C-  1)!  (37  +  C  +  1  -  j)!  (7- C  +  2 -bj)  (7- C  -  3j) 

WW-  i!(i  +  l)!(7-C-i)!(7-C  +  2-i)!(C-l-i)! 

(1.46) 

for  7  >  C. 
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This  completes  oiir  survey  of  the  traditional  uses  of  formal  lanj'uagcs  in  enumeration 
and  some  of  the  techniques  used  to  extend  them  beyond  their  context-free  limit«'>tions. 
Additional  examples  of  non-standard  use  can  be  found  in  tlic  works  of  Cori  [Cori  1970, 
1972,  and  1975],  who  uses  them  to  study  planar  graphs,  and  in  [Flajolet  1980],  where  they 
are  used  to  count  sequences  of  operations  on  data  structures. 

In  the  remaining  chapters  we  will  extend  the  usefulness  of  formal  languages  from  unla¬ 
belled  to  labelled  combinatorial  objects.  This  is  accomplished  by  adding  partial  orders  that 
govern  the  labelling  of  terminal  symbols,  and  that  preserve  the  straightforward  translation 
(of  grammars  into  e(piations)  seen  throughout  this  chapter. 

Chajitcr  2  introduces  labelled  formal  languages,  explains  the  relationship  of  grammars 
and  partial  orders,  and  demonstrates  tlie  wide  range  of  classical  generating  functions  that 
can  be  derived  from  this  new  framework.  Chapters  3  and  4  explore  some  of  the  applications 
of  labelled  formal  languages  in  diverse  areas  such  as  the  analysis  of  algorithms  and  the 
counting  and  random  generation  of  combinatorial  objects. 

The  grainmatic.al  extensions  that  follow  are  believed  to  be  new,  although  they  bear 
some  relation  to  recent  efforts  to  vest  analytic  operations  with  combinatorial  meaning  (see 
for  example  [Joyal  1981]).  The  “diminished  searcli  trees*’  of  Chapter  3  do  not  appear  to 
be  either  known  or  analyzed  by  earlier  authors,  however  the  differential  equations  used  in 
Chapter  3  arc  also  used  to  study  the  closely  related  “median  of  n’’  modification  of  quid  "ort 
[Scdgcwick  1975],  [Knnth  1975].  Finally,  the  general  purpose  system  described  in  Chapter  4 
and  implemented  in  appendix  C,  seems  to  be  unique  in  its  ability  to  generate  a  viist  variety 
of  combinatorial  objects  from  short  grammatical  descriptions. 


Chapter  2 


Labelled  Formal  Languages 

In  the  previous  chapter  we  found  that  grammars  could  express  a  variety  of  interesting 
structures  and  were  easily  translated  into  generating  functions.  However,  the  class  of 
combinatorial  objects  expressible  in  this  way  is  limited.  We  could  encode  unlabellcd  trees 
with  I'a  and  h'a  for  the  lower  and  higher  movements  of  a  tree  walk: 

xlxhlxh,  (2.1) 

but  it  was  not  possible  to  generate  labelled  trees,  such  as 

x^lxihlxih  (2.2) 


with  ordinary  grammars. 

The  purpose  of  this  chapter  is  to  define  an  extended  form  of  grammar  that  will  generate 
the  second,  labelled,  string  given  above  while  retaining  the  nice  translation  property  of 
the  preceding  chapter;  it  will  still  be  possible  to  convert  a  grammar  systematically  into 
an  equation  and  obtain  a  generating  function,  although  the  equations  will  typically  be 
difierential  and  the  generating  functions  will  always  be  exponential. 

2.1  Smallest  Label  Control 

Throughout  this  section  the  labelled  tree  problem  will  serve  as  a  good  illustration: 

Problem.  How  many  rooted  imordercd  labelled  trees  with  n  nodes  are  possible? 
(Cayley  1889}  /Moon  1970} 

Unordcred  means  that  we  do  not  care  about  the  ordering  of  subtrees  at  each  node.  With  or¬ 
dered  subtr<H»  the  answer  is  just  n!  times  the  number  of  ordered  tmlabellod  trees  computed 
in  Chapter  1,  so  it  is  the  permuting  of  subtrees  that  makes  this  an  interesting  problem. 

A  labelled  formal  language  has  two  new  features.  First,  there  is  a  special  terminal 
character  x.  The  occurrences  of  the  special  character  receive  distinct  labels  in  the  range  1 
to  n,  where  n  is  the  total  number  of  special  characters.  Second,  every  derived  string,  partial 
derivation  and  production  in  the  grammar  has  an  associated  partial  order  that  specifies 
acceptable  ways  of  labelling  the  special  characters.  For  example,  suppose  we  derive  the 
string 


xjxbhlxeh 


(2.3) 
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roprcsontinj'  a  small  tree  with  two  sons.  Here  a,  6,  and  c  arc  variables  for  the  labels  1,  2, 
an;l  3  that  have  yet  to  be  assigned.  Since  we  do  not  care  about  the  ordering  of  subtrees 
the  two  labellings 

X2lx\hlx:^h  and  X2lx:^hlxih  (2.4) 

are  redundant;  only  one  should  be  produced  by  the  grammar,  so  the  partial  order  6  <  c  is 
associated  with  the  string: 

[6  <  c]  xjxihlxch,  (2.5) 

This  way  all  labellings  of  the  string  will  be  in  a  canonical  order,  with  the  label  of  the  first 
subtree  less  than  the  label  of  the  second  subtree. 

We  must  now  expect  our  grammar  to  produce  both  a  string  and  a  partial  order.  For 
this  purpose  variables  are  added  to  any  nonterminal  that  can  derive  a  string  with  I’s: 

T-^\\x,F, 

F^\e<S]lT,hF,\€. 

We  also  associate  a  partial  order  with  each  production  possibility.  In  the  production  T 
[l^aF'b,  the  h  subscript  on  the  nonterminal  F  is  a  variable  for  one  of  the  labels  1  to  n. 
However,  since  F  can  derive  a  string  with  many  x’s,  the  h  stands  for  only  the  smallest  label 
amongst  the  x’s  derived  by  F,  if  there  is  at  least  one  x.  Thus  the  production 

F  [e  <  /]  IT.hFf  (2.7) 

requires  that  the  smallest  label  in  the  string  derived  from  be  less  than  all  the  labels  in 
the  string  derived  from  Fj.  This  constraint  is  precisely  what  we  want  for  the  tree  problem, 
since  this  production  is  spinning  off  the  descendants  of  a  node  in  such  a  way  that  the 
smallest  label  will  appear  in  the  leftmost  subtree,  and  of  the  remaining  labels  the  smallest 
will  appear  in  the  second  on  the  left,  a!id  so  on.  Notice  that  these  smallest  labels  can 
appear  anywhere  within  their  subtrees;  we  arc  not  ordering  the  roots  of  the  subtrees  as  one 
might  expect.  Nevertheless,  the  final  labelling,  if  it  obeys  the  partial  order  fragments  in 
each  production,  will  be  a  canonical  representation  of  a  tree. 

A  grammar  of  an  ordinary  formal  language  is  a  set  of  rewrite  rules.  Beginning  with 
the  “start”  nonterminal  and  repeatedly  replacing  nontermiinds  by  one  of  their  production 
possibilities  eventually  results  in  a  string  of  all  terminals.  A  sim  ^  process  works  for  la¬ 
belled  formal  languages.  In  the  tree  example  we  modify  the  grammes  slightly  by  expanding 
the  first  production,  so  that 

T-*[]xaFf,  (2.8) 

is  changed  to 

T  [o  <  6]  x^Fb  I  [c>  d]  XcFa .  (2.9) 

This  way  each  production  possibility  records  the  location  of  the  smallest  label. 

The  string  portion  of  a  labelled  formal  language  functions  like  a  rewrite  system,  with 
renaming  of  tlie  subscript  variables  a,6, c...  if  necessary  to  insure  that  all  the  subscripts 
in  cUiy  partial  derivation  are  distinct. 

The  partial  order  is  modified  by  substitution.  Suppose  we  h?vc  a  partial  derivation, 

[F]tr5ot;,  (2.10) 
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ainl  wo  apply  the  production 

5~^[C]u;,  (2-11) 

whom  u,  V  ti;  arc  strings,  P  is  a  partial  order  on  the  subscripts  of  uS^v^  and  l2  is  a 

partial  order  o:>  tlu'  s\ibsrripts  of  w.  then  S  is  rewritten  to  w,  and  a,  in  partial  order  P,  is 

replaced  by  ihe  smallest  item  of  partial  order  Q.  The  ri'^ii^aindcr  of  Q  is  also  added  to  P , 
but  with  no  additional  relationships  between  P  and  £.  sav-  those  implied  by  the  addition 
of  the  smallest  item  of  2  to  P.  In  brief,  the  application  of  (2.11)  to  (2.10)  yields: 

\P  with  Q,  substituted  for  o)uti;v.  (2‘12) 

The  substitution  of  partial  orders  is  linked  to  the  substitunon  of  strings  by  the  following 
requirement:  if  5  is  being  rewriteu  to  the  empty  string,  then  o  must  be  maximal  in  P.  Or, 
equivalently,  whenever  a  nonterminal  has  a  subscript  that  is  not  maximal  it  must  generate 
at  least  one  x. 

A  labelled  version  of  the  small  tree  example  sh  ;uld  clarify  the  substitution  process. 


r  -♦  [a  <  6]  XaFf, 

(1) 

r  -♦  (o  dj  XcFa 

(2) 

F-^  [e<  f]  IT.hFf 

(3) 

F^( 

(4) 

(2.13) 


The  productions  used  are  recorded  on  the  right: 

(ir 

[c  > 

(c>  c  <  /] xJTthFf 

|c  >  o  <  /;a  <  b]XclXaFthFf 

[c>  o  <  /] xJxahFf 

(c  >  o  <  3  <  *]  xJxahlTghFi 

[c  ^  fl  ^  ^  ijj  ^  fc]  Xc^XfihXxjFii^xFx 

\c>  a  <  j  <  »■]  xJxahlxjhFi 

(c>  o  <  j]  xjxahlxjh 


(2) 

(3) 
(1) 

(4) 

(3) 
(1) 

(4) 
(4) 


(2.14) 


Note  that  the  partijil  order  obtained  by  this  process  is  stronger  than  necessary  since 
we  only  care  tliat  a  <  j  (compare  with  cipiation  (2.5)).  Tlic  extra  strength  is  due  to  the 
producti«)ii  chosen  in  the  first  step  of  the  derivation;  using  T  — -  (o  <  6]  XaFb  would  have 
derived  another  possibility,  |c  <  o  <  y] .  Tliis  is  ultimately  a  consequence  of  our  splitting 
the  first  production,  T  -*  (JiaA,  into  two  cases  a  <b  and  b  >  a.  However,  this  splitting 
process  is  necessary,  for  in  order  to  substitute  one  partial  order  into  another  wc  must  know 
the  smallest  item  in  the  substituted  order. 

The  tree  example  illustrates  another  important  constraint.  At  several  points  in  the 
derivation  the  fourtli  ]>roductiou  was  applied  mid  labels  disappeared  from  the  partial  order. 
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i.ai]i:lli:d  formal  lanouaoks 

At  tlu'sr  ptMiits  wo  cannot  allow  tlio  lahol  <lisapprarin^;  to  bo  loss  than  any  other  lahoh  Thns 
the  rewrite  F  -♦  (  can  bo  preceded  !)y  T  [a  <  />]xaF6,  but  not  by  T  —•  [c  >  d]xcFti. 
This  requirement  eliminates  uuwantod  ambiguity. 

In  summary,  an  lab('lle<l  formal  language  describes  two  intimately  linked  actions:  a 
rewriting  process  for  strings  and  a  substitution  pr()C(\ss  for  partial  onlers.  As  the  grammar 
rowrit('s  a  string  it  weaves  together  a  partial  order  that  specifics  acceptable  labellings  of 
the  special  characters. 

2.2  Translation  to  Generating  Functions 

The  simplest  and  perhaps  most  useful  labelled  formal  languages  have  productions  of 
the  form 

i?  [6  <  g;  6  <  c;  6  <  d . . .]  SanU.Va ....  (2.15) 

where  the  partial  order  specifies  one  label,  in  this  case  6,  to  be  less  than  all  the  others. 
For  the  time  being  we  will  restrict  our  attention  to  these  simple  partial  orders,  and  use 
the  following  shorthand  notation:  a  box  superscript  marks  the  nonterminal  or  terminal 
receiving  the  smallest  label,  so  the  above  production  would  be  noted  as: 

R-*ST°UV,,.,  (2.16) 

and  the  tree  grammar  of  the  preceding  section  would  appear  as:  . 

T-^xF 
F  -*  lT°hF  I  e 

The  absence  of  a  box  on  the  first  tree  prodiiction  denotes  the  absence  of  constraints.  We 
could  also  have  written, 

T-^x°f\xF°,  (2.18) 

expanding  the  production  as  we  did  in  the  preceding  section.  In  later  sections  wc  will 
explore  more  complex  parti;J  orders. 

The  translation  of  labelled  formal  languages  to  generating  functions  proceeds  as  follows: 
1)  Convert  the  grammar  to  a  set  of  equations  by  changing 

a)  — ►  to  = 

b)  I  to  + 

c)  €  to  1 

d)  sT^uv ...  to  j{sruv . . .) 

2)  Solve  the  diffcTential  equations  for  the  start  syml)ol  and  treat  the  VQsxilt  as  a  null- 
tivariate  generating  function  that  is  exponeiiti«J  in  the  special  character,  aiul  ordinary  in 
the  other  tiTiiiinal  symbols. 

The  tree  grammar,  for  example,  translates  to 


T  =  xF 


(2.17) 


(2.19) 
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The  Vs  and  h’s  arc  not  particuhirly  interesting  to  count,  so  /  and  h  arc  set  to  1.  After 
dilferentiation  the  second  equation  becomes 

F'  (2.20) 


with  solution  F  =  c^,  so  we  obtain  the  classic,  implicit,  generating  function  for  labelled 
trees  [Polya  1037]: 

r  =  xc^ . 

Notice  that  it  doesn’t  matter  which  form  of  the  first  production  we  use  for  the  translation. 
The  first  version  T  xF,  yields  T  ^  xF  while  the  second  version,  T  -4  x°F  |  xF°,  yields 
r  =  /  x'F  +  /  xF',  but  both  arc  equivalent  under  integration  by  parts.  Conibinatorially, 
the  integration  by  parts  rule  simply  states  that  the  smallest  label  must  appear  in  one  of 
the  two  substructures  on  the  right  hand  side. 

Why  does  this  translation  work?  The  key  step  is  l.d,  where  ST°UV ...  is  changed  to 
J{STUV . . .).  The  other  aspects  of  the  translation  are  similar  to  the  procedures  used  for 
ordinary  formal  languages.  Step  l.d,  however,  integrates  the  whole  term,  with  the  derivative 
placed  on  the  boxed  terminal  or  nonterminal  witliin  the  term.  Both  the  integration  and 
the  differentiation  are  with  respect  to  the  spcci.il  character,  in  this  case  x,  which  is  now 
treated  as  a  commutative  variable.  The  constant  of  integration  is  always  zero,  that  is, 
/  f{x)  means  J*  f{y)dy. 

It  is  not  hard  to  see  why  this  works.  Suppose  we  have  a  production  R  ST  which 
translates  to  i?  =  ST,  and  we  have  two  exponential  generating  functions  for  S  and  T: 


Then  the  product  is  given  by 

where  the  inner  term,  (J)sitifc-,-,  builds  a  labelled  derivation  for  R  by  taking  any  derivation 
Si  for  5,  combined  with  any  derivation  tk^i  for  T,  and  relabelling  so  that  the  relative  orders 
of  the  labels  within  S  and  T  individually  arc  maintained,  but  the  two  sets  of  labels  are 
intermingled  in  all  possible  ways,  (J). 

Suppose  now  that  we  insist  that  the  smallest  label  appear  in  the  subtree  derived 
from  r  by  writing  R  5r°.  With  exponential  generating  functions,  the  integration  and 
differentiation  in  the  translation  of  this  production,  R  =  act  like  shift  operators: 


i>o  ^ 


(2.24) 
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Therefore  the  product, 


corresponds  to  shifting  the  smallest  label  of  T  away,  intermingling  the  remaining  labels, 
and  then  returning  the  smallest  label.  We  obtain  rather  than  (*). 

2.3  Examples 

This  section  reviews  a  variety  of  classic  enumeration  problems  in  order  to  show  the 
broad  applicability  of  labelled  formal  languages.  For  each,  problem  we  will  find  an  enco«ling, 
grammar,  translation,  and  solution.  Since  some  of  the  solutions  will  be  given  implieitly,  by 
equations  like  T  =  xe^,  a  .special  appendix  has  been  devoted  to  the  techniques  of  Lagrange 
and  Bell  for  recovering  meaningful  information  from  such  equations.  For  each  of  the  chassic 
problems  two  references  arc  given,  the  fir.st  to  the  original  source  of  the  problem,  and  the 
second  to  a  more  accessible  modern  reference. 


2.3.1  Alternating  Permutations 

Problem.  Count  the  number  of  permutations  of  1,2,  ...,n  that  obey 

>  tT2i  <  ojin  tor  all  i.  In  these  permutations  the  first  entry  is  large  anil 
thereafter  large  and  small  strictly  alternate.  [Andre  1878]  [Comtet  1074;  p.  258} 

For  the  sake  of  clarity  we  restrict  the  problem  to  odd  length  permutations.  The 
encoding  of  these  permutations  is  simply  a  string  of  I’s,  with  the  ith  x  having  label  So 
for  n  =  3  there  are  two  correctly  alternating  permutations,  ijXjis  and  isiiij. 

The  grammar  for  alternating  permutations, 


A-*Ax°a\x,  (2.26) 

is  based  on  the  observation  that  the  smallest  labelled  x  in  the  permutation  splits  the 
permutation  into  two  correctly  alternating  pieces.  The  translation  of  the  grammar, 


(2.27) 


reduces  to  a  differential  equation 

A'  =  A®  +  1  (2.28) 

with  solution  A  =  tan  x. 


Tliis  example  raises  some  interesting  questions  of  ambiguity.  In  the  precc<ling  chapter  a 
grammar  had  to  be  unambiguous  to  be  useful  for  enumeration.  The  same  holds  for  labelled 
formal  languages,  although  in  this  case  the  underlying  ordinary  grammar  A  — ►  Ax  A  |  x  is 
very  ambiguous.  It  is  the  box  operator,  the  augmentation  to  the  grammar,  that  makes  this 
an  unambiguous  grammar  and  allows  us  to  obtain  a  useful  generating  function. 
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2.3.2  Stirling  Numbers  of  the  First  Kind 

Problem.  How  many  pcTumtations  of  n  elements  have  exactly  k  cycles? 

This  time  permutations  arc  encoded  by  their  cycle  structure.  For  example,  the  permu¬ 
tation  taking  1234  to  3241  has  cycle  structure  (2)(31*i),  but  since  (2) (143)  and  (314)(2)  also 
represent  the  same  permutation  we  will  disambiguate  these  possibilities  by  writing  cycles 
with  their  smallest  element  first,  and  ;vrranging  the  cycles  so  that  the  miniinums  increase 
from  left  to  right,  c.g,,  {143)(2).  This  is  captured  with  the  grammar 

P  -  C°bP  I  € 


The  first  production  lays  out  the  cycle  structure,  with  6*8  separating  the  cycles.  The  box 
operator  insures  that  the  cycles  will  have  successively  larger  minimum  elements.  The  last 
two  productions  derive  a  cycle:  (7  — ►  x°72  makes  the  first  item  the  smallest  in  the  cycle 
and  xTZ  I  c  finishes  the  cycle  without  constraint. 

The  translation  and  solution  of  this  grammar  is  straightforward, 


-/i^ 

--=  j{C'bP)  +  \ 


P'  =  C'bP 
p  = 

giving  the  familiar  generating  function  for  Stirling  numbers  of  the  first  kind: 


=  -,)-*  =  53  My- 


Two  modifications  of  the  above  grammar  result  in  other  well  known  generating  func¬ 
tions.  By  insisting  that  no  cycle  contains  a  single  clement, 

P  C°bP  I  £ 

C  -  x°xR  (2.32) 

R^xR\€, 

we  obtain  derangements  (permutations  without  fixed  elements).  This  grammar  converts  to 


I 
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2.3.3  Stirling  Numbers  of  the  Second  Kind 

Problem.  Count  the  number  of  part iticna  of  1  to  n  into  j  nonempty  subsets.  Tins 
corresponds  to  placing  n  numbered  balls  into  j  iudistiuguishabJe  boxes,  disregarding 
the  ordering  of  balls  within  boxes. 

The  partition  is  encoded  as  a  series  of  labelled  x’s,  with  p’s  marking  the  boundary 
between  subsets.  Within  a  subset,  since  the  order  doesn’t  matter,  the  x’s  are  arranged  in 
ascending  order  of  their  labels.  Thus  there  arc  three  partitions  of  n  =  3  into  j  =  2  parts: 

xipxjxsp 

X1X2PX3P  (2.36) 

X1X3PX2P 

An  extra  p  appears  at  the  right  of  each  string  so  that  there  arc  as  many  p’s  as  blocks  in 
the  partition. 

The  grammeir, 

F  B°pF  I  f 

B  -*  x°R  (2.37) 

functions  by  first  laying  out  the  blocks  of  the  partition  arranged  in  increasing  order  of  their 
smallest  labels.  Inside  a  block,  the  last  two  productions  arrange  the  labels  in  ascending 
order,  and  insure  that  there  is  at  least  one  x  per  block. 

Beginning  with  the  last  production  we  can  transform  and  solve  the  grammar: 

R=J R+1 
R  =  e* 

B=z  Jr 

B  =  e*- I 

f^pJb'f+i 


(2.38) 

4 
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Notice  that  when  e*  is  integrated  to  find  we  must  subtract  1  to  make  the  constant  of 
integration  zero.  The  solution  gives  a  generating  function  for  the  Stirling  numbers  of  the 


second  kind, 


(2.39) 


Later  on,  we  will  find  use  for  the  associated  Stirling  numbers  of  the  second  kind, 
denoted  by  ,  and  equal  to  the  number  of  ways  of  partitioning  n  into  j  subsets  with 

at  least  s  items  per  subset.  Rewriting  the  above  grammar  to  force  larger  subsets, 


F  -►  D°pF  I  e 

B{  -*  x°Bi-i  l<i<s  (2.40) 

Bq  -»  x°Bo  I  e, 

and  translating  this  grammar,  gives  a  generating  function  for  associated  Stirling  numbers: 


K'’-  s  f) 

-  ^  0<t<»  ' 


F^e 


2.3.4  Mappings 

A  mapping  of  the  integers  1  to  n  into  itself  will  consist  of  several  disjoint  components, 
each  with  a  central  cycle.  Pictorially,  a  mapping  looks  something  like  this: 


Figure  2.1  A  Mapping  of  1  to  11  into  the  Same  Range 

Problem.  How  many  mappings  /  :  {1 . . .  n}  — ►  {1 . . .  n}  have  exactly  j  components? 

If  a  mapping  has  only  one  loop  and  no  other  cycles  then  the  mapping  is  a  labelled  tree. 
On  the  other  hand,  a  surjection  will  consist  entirely  of  cycles,  with  no  treo-like  structure. 
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The  grammar  for  ina])piBgi5  combines  these  two  extremes:  it  includes  a  both  a  tree  generator 
and  pernuitation  generator: 

P  -*  C°bP  I  e 
C-*T°R 

R-*TR\€  (2.42) 

T-*xF 

F-^lT°hF\€. 

The  first  three  productions  have  already  appeared  in  Section  2.3.2  on  permutations  and 
Stirling  numbers  of  the  first  kind,  and  the  last  two  productions  form  a  familiar  tree  genera¬ 
tor.  Together  these  productions  generate  mappings,  coded  with  6*s  separating  components 
of  the  mapping,  and  lists  of  trees  within  components  ordered  so  that  the  tree  roots  form 
the  cycles.  The  mapping  pictured  above  would  encode  as: 

x^lxglxihlxshhx2lxihxQbxzXiibxiolx5hb,  (2.43) 


Portions  of  the  solution  of  this  grammar  are  given  implicitly, 

r  =  xe^ 

R  =  (2.44) 

C  =  -ln(l-r) 

p  =  g“tln(l-T) 


so  we  don’t  have  a  closed  form  for  P,  However,  using  the  techniques  of  Appendix  A, 
detailed  information  can  be  recovered  from  the  generating  function: 


Metropolis  and  Ulam  initiated  the  study  of  the  number  of  components  in  a  random 
mapping  with  some  empirical  residts  [Metropolis  1953].  Subsequent  authors  were  able  to 
compute  the  expected  number  of  components  and  give  complex  formulas  for  the  distribu¬ 
tion.  The  comparatively  tight  expression  in  (2.45)  was  discovered  by  Riordan  [Riordan 
19C2|.  Other  questions  about  random  mappings  such  as  the  number  of  recurrent  elements 
(those  involved  in  cycles)  have  been  studietl  land  are  also  anieiiablc  to  treatment  with  la¬ 
belled  formal  languages. 


SCHRODER'S  THIRD  PROBLEM 


25 


2.3.5  Schroder’s  Third  Problem 

Problem.  A  set  of  n  hhcllcd  elements  is  chciiRcd  together  in  groups  of  size  m  as 
follows:  we  repeatedly  collect  m  eJenjenfs,  delete  them  from  the  set  and  then  add 
them  hack  together  as  a  singlCy  new,  grouped  element  until  only  one  element  remains 
in  the  set.  This  single  element  is  the  root  of  a  labelled  m-way  branching  tree.  How 
many  such  trees  or  chmnings  arc  there?  [Schroder  I870j  [Comtet  1974;  p.  165] 

The  problem  is  solved  with  a  modified  tree  grammar,  designed  to  force  exactly  m 
descendants  at  each  node: 


T  -►  bSmC  I  X 

l<i<m  (2*46) 


A  matched  pair  b  (begin)  and  e  (end)  mark  the  left  and  right  ends  of  chains.  So,  for 
example,  there  arc  ten  chains  of  n  =  5  elements  with  grouping  factor  m  =  3: 


66x1X2X36X4X5^ 

66x1X2X46X3X56 

66x1X2X56X3X46 

66x1X3X46X2X56 

66x1X3X56X3X46 


66x1X4X56X3X36 

6x16x3X3X46X56 

6x16x3X3X56X46 

6x16x3X4X56X36 

6x1x36x3x4x566 


(2.47) 


Notice  that  since  the  problem  specifies  no  order  among  the  m  terms  in  a  chain,  the  grammar 
uses  a  box  operator  to  specify  a  canonical  left  to  right,  smallest  to  largest  layout. 

Dropping  b  and  c  from  the  problem,  the  grammar  translates  to: 


T=S„.  +  x 

Si  =  j  T'Si-i  1  <  »•  <  m  (2.48) 

Si  =  T, 


from  which  we  conclude  that 


"  i! 

(2.49) 

and  _ 

r=I+— r. 

m! 

(2.50) 

Once  again  the  generating  function  is  given  implicitly  and  the  techniques  of  Appendix  A 
are  applicable: 


-  l)/(Tn-  1))  (ml) 


J 


(2.51) 


when  m  -  1  divides  n  —  1. 
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2.3.6  Schroder’s  Fourth  Problem  and  Scries -Parallel  Networks 

Problem.  A  Schroder  system  is  a  collection  of  subsets  of  the  integers  1  to  n  that 
inchidcs  each  singleton  subset  {»},  the  whole  set,  {1,2,3, . . and  other  subsets  that 
arc  strictly  hierarchical,  that  is,  A  C  D,  D  C  A,  or  An  D  =  0.  VVe  wish  to  count  the 
number  of  Schroder  systems  of  n  elements.  [Schroder  ISTOj  [Comtet  74;  p.  224] 

Since  they  are  hierarchical,  Schroder  systems  can  be  generated  with  labelled  formal 
languages.  The  terminals  b  and  e  are  used  to  bracket  each  subset: 

S  ->  6a:c  I  bS°Me 

M  -*  S°Q  (2.52) 


Notice  that  S  — ♦  bxe  encloses  every  integer  in  its  own  snbset.  The  combination  of  produc* 
tions  S  and  M  S°Q  eliminates  redundcint  subsets  like  bbbxieee. 

Working  backwards  through  the  grammar,  the  productions  convert  to  equations  that 
are  readily  solved: 


<3  =  /s'(3  +  i 

«  =  .« 

A/  =  /  S'Q 

J  (2.53) 

Af  =  e®  -  1 

S  =  bex  +  J  S'M 

S  =  bex  +  be  (e®  —  S  —  l)  . 

For  counting  purposes  b  cind  e  are  redundant—  one  of  them  can  be  dropped  from  the  equa- 
tion.  Applying  Lagrange  inversion  to 

5  =  6x  -h  6  (e^  -  5  ~  l)  (2.54) 

gives  an  expansion  in  terms  of  the  associated  Stirling  numbers  of  the  second  kind: 


tri  j  i/j 


Networks  of  resistors  motivate  a  problem  that  is  closely  related  to  Schroder  systems. 
Every  resistor  has  a  different  label,  and  the  ordering  within  a  scries  or  parallel  group  is 
irrelevant.  So,  for  instance,  there  arc  only  eight  significantly  different  networks  of  three 
labelled  resistors: 


SCimODER’S  FOURTH  PROBLEM  AND  SERIES-PARALLEL  NETWORKS 


27 


Figure  2.2  All  Eight  Networks  of  Three  Resistors 

Problem.  Coimt  the  nuiuber  of  distinct  networks  of  n  resistors. 

A  network  is  encoded  by  using  p  and  q  (backwards  p)  to  bracket  a  set  of  circtiits  in 
parallel,  and  s  and  z  to  analogously  bracket  circuits  in  series.  The  string  between  a  matched 
p  and  q  can  contain  individual  resistors,  x*,  or  series  circuits  enclosed  in  s  and  z.  So  the 
above  eight  networks  of  three  resistors  would  encode  as: 

PX1X2X37  SXiXjXsZ 

spxiX^qxzz  spXiXzqx2X  sxipx^Xzqz  (2.56) 

psXiXzZXzq  psXiXzzxzq  pxisx^xzzq 

Needless  to  say,  there  is  a  great  deal  of  symmetry  between  scries  and  parallel.  The  tlirce 
productions  that  generate  a  parallel  circuit, 

A  — ►  X  I pE^B 

B  E^C  (2.57) 

C-^E°C\q, 

are  identical  to  the  three  productions  for  series  circuits, 

E-*x\  sA°F 

F  A°G  (2.58) 

G^A°G\z. 

To  this  wc  might  add  an  initial  production, 

T-*A\e,  (2.59) 

that  allows  the  whole  network  to  be  series  or  parallel,  but  unfortunately  this  production 
generates  two  different  single  resistor  networks;  everything  is  fine,  except  for  the  case  n  =  1. 
To  remedy  tins,  wc  advance  the  first  production  beyond  A  and  E^ 

T-*x\pE°d\3A°F, 


(2.60) 
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and  inclu(]c  only  one  copy  of  i. 

When  translating  this  grammar,  A  can  be  expressed  in  terms  of  E, 

A  =  x+p(e^  -  E  -  l)  , 
and  likewise,  E  can  be  expressed  in  terms  of  A, 


(2.61) 


E  -  x  + s{e^  -  A- 1)  .  (2.62) 

Since  we  are  seeking  the  total  number  of  c’rcuits  of  ti  resistors,  independent  of  the  number 
of  internal  series  and  parallel  constructs,  we  need  only  invert  E  =  x  +  e^-E  —  Ito  obtain 


(2.63) 


Except  for  the  case  n  —  1  the  number  of  series-  parallel  networks  is  twice  this  coefficient, 


(2.64) 


The  connection  with  Schroder  systems  should  now  be  apparent.  A  Schroder  system 
becomes  a  series -parallel  network  when  it  is  “striped,”  by  choosing  a  series  or  parallel  nature 
for  the  largest  set,  and  then  alternating  series  and  pardlel  down  the  chains  of  successively 
smaller  subsets. 

The  stiKly  of  networks  was  begun  by  MacMahon  [MacMahon  1892],  who  gave  a  for¬ 
mula  for  the  unlabelled  problem.  Knodel  found  the  implicit  generating  function  for  the 
labelled  problem  [Knbdel  1051]  and  Garlitz  and  Hiordan  noticed  the  corresj)oudcnce  with 
Schroder  s  problem  [Carlitz  1959].  A  good  exposition,  combining  the  labelled  and  unla- 
belled  variations,  can  be  found  in  (Riordan  1978]. 

2.3.7  Eulerian  Numbers 

Problem.  A  descent  in  a  pcrnnttntJon  is  a  pair  of  ndjneent  elements  such  thnt  0%  > 

We  wish  to  count  the  number  of  permutations  with  j  descents. 

For  this  problem  we  encode  the  permutation  tlirectly  in  the  labels  of  the  x'a.  A  g 
is  inserted  betwei-n  every  pair  of  I’s  with  decreasing  labels.  For  example,  there  are  four 
permutations  of  n  =  3  elements  with  j  =  I  descent: 

xixsgxi  X2gxixs 

XiXjgxi  xzgxiXi  (2.65) 


The  grammar  for  this  encoding  relics  on  a  simple  fact:  the  smallest  label  will  always 
cause  a  descent,  unless  it  is  at  the  left  end  of  the  permutation: 

E  -*  Egx°E  I  x°E  |  Egx°  1 1.  (2.66) 
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The  differential  equation  derived  from  this  grammar, 

=  gE^  +  ^  +  qE  +  ). ,  (2.67) 

is  complicated  by  the  nonlinear  function  of  E  on  the  right  side  of  the  equation.  Since  this 
type  of  equation  has  appeared  before  (with  alternating  permutations,  Subsection  2.3.1),  we 
pause  now  to  consider  a  solution  strategy. 

If  the  right  side  of  the  equation  has  no  constant  term, 

y'  =  JfciK^  +  Jtay,  (2.68) 

then  the  transformation  y  =  1/Z  yields  a  first  order  line2U*  differential  equation, 

=  +  (2-69) 

When  a  constant  term  is  present  in  the  original  problem, 

y'  =  ikiy^  +  ifc2y  +  fc3,  (2.70) 

then  a  constant  c  is  also  added  to  the  tr^msformation, 

K  =  |  +  c,  (2.71) 

giving  an  equation 

-Z'  =  iki(l  +  cZ)^  -f  fc2(l  +  cZ)Z  +  kzZ^  .  (2.72) 

First  kic^  +  k^c  +  fca  =  0  is  solved  for  c  to  eliminate  the  Z^  term  from  the  right  side  of  the 
equation,  and  to  reduce  the  problem  to  a  first  order  linear  differential  equation. 

Returning  to  the  problem  of  descents,  we  find 

(2.73) 

gc\  +  (g  +  l)ci  +  1  =  0  (2.74) 

c,  =  -l  or  --  (2.75) 

9 

(Curiously,  the  choice  of  root  here  does  not  matter,  so  ci  =  -1  is  used.) 

-Z’  =  g-2gZ  +  {9  +  l)Z 

Z'  =  (g-  1)Z  -  g  |2.76) 

Z  =  -J—  + 

g-  1 

Since  the  grammar  does  not  generate  an  empty  string,  E{0)  is  zero,  ^(0)  is  one  and  so 
Cj  =  -l/{g  -  1).  Assembling  the  results,  wc  obtain  the  classic  generating  function  for 
Eulerian  numbers  [Euler  1755;  p.  487): 

E= 

g-gX{9-l) 


(2.77) 
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A  recurring  theme  of  the  j)rece(ling  examples  is  the  idea  of  laying  out  a  structure 
in  canonical  form.  A  group  acts  on  some  portion  of  the  structure,  and  yet  judicious  use 
of  the  l)ox  operator  insures  that  only  one  representative  from  each  eqtiivalence  class  is 
generated  by  the  grammar  and  counted  by  the  generating  function.  So  far  weVe  seen 
tlie  symmetric  group  (acting  on  such  things  as  the  descendants  of  nodes  in  trees,  and 
collections  of  indistinguishable  subsets)  and  the  cyclic  group  (acting  on  the  cycles  within 
permutations).  To  these  two  groups  we  can  add  two  closely  related  groups,  the  alternating 
group  and  the  dihednd  group.  The  use  of  the  box  operator  with  these  four  wtII  known 
groups  is  summarized  below.  The  elements  of  the  group  are  denoted  by  E's  which  can  be 
cither  single  x's  or  nonterminals  that  derive  structures  containing  x’s. 

1)  The  Symmetric  Group,  Sn' 

S^E°S\€.  (2.78) 

Here  the  elements  of  the  group  are  arranged  in  sorted  order,  based  on  the  smallest  label 
within  each  element. 


2)  The  Alternating  Group,  A,j.  The  grammar  must  permit  an  additional  degree  of 
freedom  in  the  last  two  elements: 

S-*E°s\EE.  (2.70) 


3)  The  Cyclic  Group,  C„. 
the  first  element, 


A  cycle  is  in  canonical  form  when  the  smallest  label  is  in 


5  E^T 
T-*ET\€. 


(2.80) 


4)  The  Dilicdral  Groiip,  D„.  The  dihedral  group  is  laid  out  in  the  order  depicted 
below; 


4 


Figure  2.3  Layotit  for  the  Dihedral  Group 

where  the  group  is  generated  by  a  ryclk  shift,  (1357 . . .  C42),  ,uul  a  Hip,  (1)(23)( 45)(G7) . . .. 
The  grammar  jilares  the  smallest  label  in  the  first  clement  and  insists  that  the  second 
clement  contains  a  smaller  label  than  the  third: 

S  E'^TU 

T-*E°E  (2.81) 

U-*EU\€. 

In  this  way  the  four  cunimon  groups  can  be  encoded  with  labelled  formal  languages. 
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2.4  A  Gcncra)i*pd  Definition 

From  .1  bottom-up  perspective,  the  lal)elle<l  production  C  — ♦  AD  constructs  a  string  C 
by  first  constructing  two  labelled  strings.  A  and  D,  then  concatenating  the  strings;  and  then 
shuffiing  the  labels  together,  in  much  the  same  way  that  two  decks  of  canls  arc  shuffled 
together.  The  presence  of  a  box  operator,  C  -*  A^D  constrains  the  shuffle  so  that  the 
smallest  label  in  A  is  also  the  smallest  label  in  C\  one  c.ard  is  flipped  down  from  the  A  deck 
at  the  bc'^inning  of  the  shuffle. 

Pursuing;  the  Ccird  analogy  furtlirr,  a  ‘‘gooeP  dealer  can  control  more  than  just  the  first 
card  on  the  bottom  of  the  deck.  The  intermingling  of  the  first  few  cards  as  well  as  the  last 
few  cards  can  be  manipulated  by  the  dealer  as  he  shuffles.  In  the  definitions  that  follow, 
two  partial  orders  are  inti  jduced  in  the  grammars.  The  partial  order  enclosed  in  brackets 
constrains  the  mingling  of  the  smallest  labels  while  the  p<irtial  order  in  braces  controls  the 
largest  labels.  So,  for  instance,  the  production 

c  -  {c>  /  >  3}  [6  <  C  <  a]  Af„Y>  (2.82) 

spt'cifies  that  the  two  largest  labels  will  appear  in  the  string  derived  from  A  and  the  two 
smallest  labels  will  be  in  ZJ's  derivation. 

Before  wc  attack  the  general  definition  of  labelled  formal  languages  wc  need  some 
basic  facts  about  partial  orders.  It  is  helpful  to  have  a  definition  of  partial  orders  that 
distinguishes  between  relations  and  sets: 

Definition.  A  poset  is  a  set  S  together  with  a  relation  S  sitch  that: 

1)  aSb  =>  -^bSa  (antisymmetric) 

2)  aSb  A  bSc  =»  aSc  (transitive) 

S  is  usually  described  as  a  series  of  inequalities  amongst  the  elements  of  *9,  such  as 
a  <  6,  6  <  c.  The  transitive  closure  of  these  inequalities  is  S  itself. 

Definition.  A  linear  embedding  of  a  partial  order  is  a  mapping  m  of  S  into  the 

integers  siicli  that  aSb  m{a)  <  m(6). 

We  need  to  carefully  separate  tho.se  labels  that  will  be  arranged  according  to  a  partial 
order  and  those  labels  that  will  be  ‘'shuffled”  in  a  less  constrained  way.  For  this  purpose 
we  introduce  two  new  notions:  active  dements  and  boundary  elements.  Active  elements 
will  be  counted  with  a  partial  order,  while  boundary  elements  will  be  “shuffled.”  They  may 
appear  in  a  partial  order,  but  it  is  only  to  mark  the  c<lgc  of  the  active  elements.  For  Mie 
smallest  labels  we  have  the  following  definition: 

Definition.  An  rirnn^nt  /?  is  a  boundary  cioinrj  t  if  there  are  no  elements  a  > 

(In  the  definitions  that  follow,  the  reader  can  stipply  analogous  definitions  for  the 
largest  labels.)  For  a  production  like 

9  [fr  <  c  <  o]  Aia]Biu]  1  (2-83) 

a  is  a  boundary  clement  while  6  and  c  are  not.  Since  the  subscripts  on  a  nonterminal 
represent  the  smallest  few  labels  wc  could  extend  tlu'se  subscripts  through  more  elements. 
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Ill  t!'?  la.'t  produv  ti(Mi  wo  could  write  B  with  b  <  c  <  d.  In  this  ease  d  is  also  a  boundary 
('lenient*  but  to  avoid  addinj^  d  to  the  subscripts  of  D  wt  will  usually  write  something  like 
equation  (2.83)  and  say  that  B  has  an  implicit  boundary  chnnent. 

Definition.  An  element  is  active  if  it  is  less  than  every  explicit  or  implicit  boundary 
element. 

Thus,  in  the  example  above,  b  and  c  are  active  elements. 

Definition,  A  parfia/  order  is  weJJ  separafed  if  all  elements  arc  either  active  or 
boundary. 

Again,  our  example  above  is  well  separated.  Well  separateness  is  crucial  to  the  trans¬ 
lation  of  grammars  into  integral  equations.  We  will  use  differentiation  and  integration  to 
remove  lactivc  elements  and  count  them  according  to  partial  orders. 

For  purposes  of  derivation  however,  we  need  to  be  able  to  substitute  partial  orders. 
When  a  string  contains  a  nonterminal  like  ^nd  we  wish  to  rewrite  the  nonterminal, 
we  must  identify  tlie  two  smallest  elements  in  the  derivation  of  D.  This  is  possible  if  D  has 
a  production  D  [A],.*  where  A  is  2-smallest  determined: 

Definition.  A  k-smallcst  determined  poset  has  k  elements,  01,02,03, ...  ,ajt  such 
that  oi  <  02  <  03  ■  •  •  <  Ofc  and  all  other  elements  in  S  arc  greater  than  ojt. 

For  purposes  of  the  definition  we  include  any  implicit  bo\lnd^l^y  points.  This  insures 
thcat  01,02, ...  ,01:  will  be  less  than  all  labels,  even  those  not  involved  in  the  partial  orders. 

Definition.  (Suhstitufion  of  Partial  Orders)  Let  Q  beak  smallest  detertnined  partial 
order  ov  r  a  set  Q  with  k  smallest  elements  gi  <  q2  <  Q3  <  •••  <  gh-  Let  P  be  a 
partial  order  over  P  with  k  (not  necessardy  smaWestj  c/ements  pi  <  p2  <  P3  <  •  •  *  < 

Pk‘  Then  the  siihstitution  of  Q  for  Pi,P2>P3» •  •  •  »pjfc  in  P  is  a  partial  order  P  over 
(P  “  {pi,P2, . . .  ,Pjfc})  U  Q  consisting  of  P,  Q,  and  some  extra  patching  relations: 

1) Pi<peP=>gi<peP 

2)  Pi  >peP=>gi>peP  (gi  acts  like  pi) 

2)  p<  Pi  €PAgi<q£Q=>p<gGp  (tr<uisitive  closure  across  the 
identification  g,  ~  Pi) 


Postponing  for  the  moment  the  definition  of  label-controlled  strings  and  derivation 
steps,  we  can  define  . . . 

Definition.  A  labelled  grammar  is  a  Rvc-Utple  (T.x,iV,  5,  P)  consisting  of 

1)  a  terminal  aJphahet  T, 

2)  a  special  symbol  x  €  T, 

3)  a  nonterminal  alphabet  IV, 

4)  a  .start  symbol  5  6  IV, 

5)  a  set  of  productions  P  of  the  form 
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C  a  hibcl-Ci  .Avllcd  string  ,  (2.84) 

where  C  £  N. 

A  labcUcontrollcd  string  D  is  derived  by  the  grammar  if  it  is  possible  to  begin  with  S 
and  ohfain  D  after  a  finite  nunibcr  of  derivation  steps. 

Definition.  A  label<ontroUe<i  string  is  a  string  over  the  same  alphabet  as  the  for- 
mal  langimge  (Ny  x,  Tj  where  each  norifermina]  has  a  (possibly  empty)  collection  of 
superscripts  and  subscripts.  The  string  is  prefixed  with  two  partial  orders,  otic  over 
the  superscripts  (enclosed  in  bracesj  and  the  other  over  the  subscripts  (enclosed  in 
brackets).  The  occurrences  of  Uie  specml  cJiaracter  x  in  the  string  have  at  most  one 
superscript  and  one  subscript.  If  two  scripts  are  present,  then  one  must  be  a  boimdary 
element. 

By  insisting  that  at  least  one  label  of  a  doubly  labelled  x  be  boundary,  the  last  definition 
prevents  the  two  partial  orders  from  interfering  with  one  another. 

Definition.  (Derivation  Step.j  The  label-controlled  string 

derives  in  one  step  (C)  wGv  if: 

1)  There  is  a  production  F  {12}  [8]  G.  where  and  B  are  partial  orders 
on  the  superscripts  and  subscripts  of  G. 

2)  Either 

a)  G  has  no  x%  J  <  1,  h  <  1,  and  pi,  Oi  arc  boundary  elements  of  P 
and  A.  Ji  is  then  the  result  of  deleting  pi  from  P,  and  C  is  the  result  of 
deleting  ai  from  A. 

b)  Q  is  j  largest  determined  and  P  is  the  result  of  substituting  Q  for 

P-  8  is  k  smallest  detcruiineil  and  C  is  the  result  of  substi¬ 
tuting  8  for  ai, a2, . . . , ajb  in  A. 

As  the  definition  is  writen,  possibility  2.a  is  the  only  way  that  labels  can  disappear 
from  partial  orders.  Tlxis  is  adequate  for  the  definition,  but  we  will  sometimes  find  it 
convenient  to  have  F  derive  a  string  of  all  terminals  several  of  which  arc  special  characters. 
The  essential  point  is  that  only  boundary  elements  can  disappear  from  partial  orders. 

Definition.  A  fiill  derivation  is  a  mapping  of  the  labels  of  a  label-controlled  string 
(containing  only  terminal  symbols}  onto  the  integers  1  to  n.  The  mapping  must  be  a 
linear  embedding  of  both  of  the  parthil  orders  associated  with  the  string. 
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Wo  turn  now  to  thr  transformation  of  labollod  granunars  into  integral  equations,  for 
which  wo  will  nerd  to  identify  the  active  dements  in  the  partial  orders. 

Definition.  The  transhition  of  a  string,  {P}[A]^y  with  well  separaferf  posets  P 
and  Ay  is  the  product  of 

1)  The  number  of  linear  embeddings  of  the  active  elements  of  P . 

2)  The  number  of /incar  embeddings  of  the  active  dements  of  A> 

3)  The  integral  repeated  as  many  times  ns  there  are  active  elements  in  P 
and  Ay  of  the  product  of  the  symbols  in  w,  each  di/fi^renfiated  as  ni/iiiy  times  as 
there  arc  active  elements  in  their  subscripts  and  superscripts. 

Definition.  The  translation  of  a  grammar  is  obtained  hy  summing  for  every  nonter¬ 
minal  V  E  N  the  translation  of  each  production  possibility  for  V,  and  setting  this  sum 
equal  to  V. 

For  example,  the  production 


T  -*  (>J]  .Sjo4]i[c)5[j«] , 


(2.86) 


with  partial  order 


has  two  boundary  elements,  b  and  e,  and  three  active  elements,  a,  c,  and  d,  so  the  production 
possibility  is  well  separated.  The  translation  is 


r  =  c  III  (s')\ 


(2.88) 


The  above  definitions  make  two,  somewhat  different,  demands  on  the  partial  orders 
used  in  the  grammars.  A  <lerivation  step  requires  that  the  partial  orders  be  Jt-smallest 
or  largest  determined  in  order  to  make  siibstitutions.  On  the  other  hand,  the  process  of 
translation  to  integral  equations  requires  thcit  the  partial  orders  he  well  separated.  In  fact, 
the  original  gramincir  can  ho  presented  in  a  way  tlmt  inet'ts  neither  of  these  rcqtiircmcnts. 
If  necessary,  we  can  always  express  an  aberrant  partial  order  as  the  union  of  several  valid 
partial  orders,  as  we  did  with  the  tree  example  in  equation  (2.0). 
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2.4.1  Left  to  Right  Maxima  and  Minima 

Problem.  How  many  permufAtions  of  1  to  n  have  j  left  to  right  maxima  and  k  left 
to  right  minima? 

Pernuitations  are  encoded  in  the  subscripts  of  the  x’s,  with  an  /  before  each  maximum 
and  s  before  each  minimum.  For  example,  there  arc  six  strings  of  length  n  =  3  with  j  =  2 
maxima  mid  fc  =  2  minima, 


slxzsxiX2lx4 

slxzSXilXiX^ 

S/X3/X4SXXX3 


8lX2lX4XzSXi 

slx2lX4SXiX^ 

3lX2SXilX4X^ 


The  grammar, 


r  {c  >  /;  e  >  (7}  [6  <  a;  6  <  c] 
T  {f  >  €\  f  >  g}la  <  b\a  <  c] 


iZ  xiZ  I  c, 

is  probably  clearer  if  we  adopt  the  convention  that  ■  marks  the  location  of  the  largest  label: 

T  r*5x°iz 

T  -*  six 
iZ  — ►  i/Z  I  « . 

Permutations  are  decomposed  based  on  the  location  of  their  smallest  and  largest  elements. 
The  first  production  for  T  covers  situations  where  the  smallest  element  is  to  the  right  of 
the  largc.st  dement;  the  smallest  will  be  a  minimum,  so  it  is  prefixed  with  an  s.  The  second 
T  production  handles  those  cases  where  the  largest  is  right  of  the  smallest.  R  generates  an 
arbitrary  string  of  I’s,  for  it  is  to  the  right  of  both  the  smallest  and  largest  dements  and 
so  can  have  no  fiirthcr  maxima  or  minima. 

The  partial  orders  are  well  separated,  so,  using  the  translation  defined  above,  we 
discover 

T  =  11  (T'sR)  +11  (riR)  +  six 

R  =  xR  +  i 
1 


]nr  =  -{s  +  l)ln(l-x)  +  Ci 


/ 
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Wc  conclude  that  =  si  Fince  the  only  string  with  a  single  x  has  both  a  maximum  and  a 
minimum  at  that  x.  The  grammar  cannot  derive  an  empty  string,  so  there  is  no  constant 
term  in  the  final  generating  function: 

'+*-!) .  (2.93) 


To  derive  strings  representing  permutations  wo  need  partial  orders  that  axe  all  1- 
smallest  and  1-largcst  determined.  This  is  not  true  of  the  last  production  of  (2.90),  but  the 
failure  is  easily  remedied;  we  expand  the  production  into  several  possibilities: 

Now  every  production  is  1-sniallest  and  1-largest  determined.  Wc  can  work  a  short  example 
by  starting  with  the  first  production  of  (2.90), 

{c>  /;  c>  <;}  [6  <  a;  6  <  c]  ,  (2.95) 

deriving  a  six  from  the  T  nonterminal, 

{e  >  f\e  >  g}\b  <  a\b  <  c\  ,  (2.96) 

and  then  expanding  R  with  the  production  R 

{e  >  /;  e  >  g}  [6  <  o;  6  <  c]  .  (2.97) 

There  is  only  one  way  to  label  the  remaining  string  that  is  consistent  with  both  partial 
orders; 

slxzsxixi .  (2.98) 

The  fact  that  there  seems  to  be  one  possibility, 

R-*{j>  k}  [/»  <  i\  ,  (2.99) 

missing  from  the  productions  for  R  is  extremely  illustrative.  First,  notice  that  this  possi¬ 
bility  is  not  a  valid  production.  The  special  character  x  has  two  labels,  neither  of  which  are 
boundary  elements  of  their  respective  partial  orders.  Second,  notice  that  of  all  the  possibil¬ 
ities  for  R  at  (2.94),  only  the  invalid  production  given  here  at  (2.99)  can  be  followed  by  the 
replacement  iZ  c,  since  i  and  k  are  both  boundciry  elements.  In  fact,  if  wc  apply  R  € 
to  (2.09)  we  obtain  the  correct  production  R  found  in  (2.94).  So  the  rewriting  of 

R  to  meet  the  l-dctcnninedness  requirements  is  hardly  very  mysterious  after  aU. 


CHAPTER  3 


Analysis  of  Algorithms 


In  the  past  decade  there  has  been  rapid  growth  in  a  subspecialty  of  theoretical  com- 
ptiter  science  called  analysis  of  algorithms.  Research  has  focused  on  the  careful  computation 
of  average  and  worse  case  running  times  for  algorithms.  This  is  to  be  distinguished  from 
matbematia'il  theory  of  computatioDj  which  is  more  coiicemcd  with  the  logic  and  correct¬ 
ness  of  algorithms,  and  computational  complexity,  where  the  analysis  is  aimed  at  broader 
classifications  like  separating  O(n^)  algorithms  from  0{nlnn).  By  contrast,  auaJysis  of 
algorithms  endeavors  to  find  exact  or  detailed  asymptotic  expansions  for  the  performance 
of  algorithms.  On  the  theoretical  side,  this  attention  to  detail  leads  to  a  great  variety  of 
interesting  combinatorial  mathematics,  while  on  the  practical  side  it  can  determine  such 
questions  as  when  an  O(nlnn)  algorithm  surpasses  an  0{n^)  algorithm.  In  this  way  the 
analysis  of  algorithms  spans  both  the  theoretical  and  practical  worlds  of  computer  science. 

Figure  3.1  below  is  a  rough  diagram  of  the  process  of  analyzing  an  algorithm.  There 
are  two  paths.  Across  the  bottom  is  what  might  be  called  the  direct  approach:  within 
the  algorithm  a  quantity  of  interest  is  identified,  a  recurrence  relation  is  derived,  and  the 
recurrence  is  solved,  using  manipulations  of  discrete  mathematics.  The  other,  seemingly 
circuitous  path  through  analytic  equations  and  generating  functions  is  often  the  easiest 
method  of  solution,  because  once  recurrence  relations  are  converted  to  functional  equations 
the  solution  strategy  is  usually  routine;  we  are  solving  quadratic  or  differential  equations 
rather  than  manipulating  sums  of  binomial  coeflBcients,  and  so  the  tecliniques  of  real  anal¬ 
ysis  can  be  effectively  applied  to  discrete  problems. 

Labelled  formal  languages  are  useful  for  shifting  from  the  discrete  domain  to  the  con¬ 
tinuous  domain  of  real  analysis.  We  have  explored  already  in  Chapter  2  the  conversion  of 
grammars  to  differential  equations,  this  is  the  transition  in  the  upper  left  hand  comer  of 
Figure  3.1  and  imdoubtedly  the  simplest  step  in  the  whole  diagram.  However,  the  analyst 
needs  to  express  some  quantity  of  interest  in  the  algorithm  using  a  formal  language.  This 
requires  cleveniess-  it  is  not  an  automated  step— but  since  the  language  of  expression  is 
based  on  fonnal  languages  it  is  a  natural  means  of  expression  for  computer  scientists. 
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Labelled  _  _  .  Anal^/tic  _ _ Generating 

Grammars  ^  Equation  ^  Function 

A  I 


Algorithm 


> 


Recurrence 

Relation 


V 

Solution 


Figure  3.1  A  Simplified  Picture  of  the  Analysis  of  an  Algorithm 

The  purpose  of  this  chapter  is  to  demonstrate  the  usefulness  of  labelled  formal  lan¬ 
guages  in  the  analysis  of  algorithms.  To  some  extent  this  has  been  demonstrated  already 
in  Chapter  2,  since  variations  of  the  classic  generating  functions  appear  frequently  in  the 
analysis  of  algorithms.  To  press  the  point  further,  the  examples  chosen  for  this  chapter  use 
labelled  formal  languages  in  a  central  point  of  their  analysis,  and  use  them  in  a  way  that  is 
more  complex  than  the  examples  of  Chapter  2.  We  will  study  cache  memories  and  search 
trees. 

3.1  The  Degree  of  Associativity  of  a  Hardwired  Cache  Memory 

Because  a  computer  program  is  likely  to  make  frequent  reference  to  a  small  subset  of 
its  total  address  spare,  computer  designers  have  discovered  that  substantia]  cost  savings 
are  possible  with  memory  hierarchies.  The  idea  is  to  put  a  frequently  accessed  subset  of  the 
address  space  into  a  small,  fast,  and  expensive  memory.  Most  memory  references  will  reside 
in  this  cache  memory,  so  the  performance  of  the  computer  will  be  similar  to  a  computer 
whose  whole  memory  is  built  in  this  fast  expensive  manner,  while  the  cost  of  the  computer 
will  in  fvact  be  biised  on  a  slower,  chcai)er  main  memory. 

Fitting  the  whole  address  space  into  a  small  cache  is  difficult:  one  approtach  is  to  use  a 
fully  associative  cache  that  stores  both  the  address  and  the  content  of  each  main  memory 
entry  residing  in  the  cache.  The  cache  is  capable  of  simultaneously  comparing  a  requested 
address  with  the  addresses  of  each  of  the  entries  in  the  cache.  If  the  requested  address 
in  present  in  the  cache  its  contents  are  made  available  for  processing,  otherwise,  a  slower 
process  fetches  the  entry  from  main  memory. 

A  second  approach  to  caching  saves  most  of  the  hardwfirc  lUTi'ssary  for  parallel  com¬ 
parisons.  Part  of  the  memory  address  is  used  as  an  index  into  a  table.  Within  a  particular 
slot  of  this  table  several  memory  entries  arc  stored  along  with  the  remainder  of  their  mem¬ 
ory  addresses.  To  search  this  table  for  a  requested  address  wc  find  the  appropriate  slot, 
read  the  entire  slot  into  a  special  area  and  search  the  slot  associatively  for  the  desired  entry. 
This  is  pictured  in  Figure  3.2. 
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Address  Contents 


Contents 


Figure  3*2  A  Two  Way  Associative  Cache  Memory 

The  number  of  entries  in  each  slot  of  the  cache  is  the  degree  of  associativity.  It 
is  a  design  parameter  affecting  both  the  performance  of  the  cache,  that  is,  how  close  it 
comes  to  a  perfectly  associative  scheme,  and  the  cost  of  the  cache  since  it  determines  the 
amount  of  comparison  hardware  needed.  Empirical  studies  indicate  that  very  little  is  gained 
by  increasing  the  associativity  beyond  2  or  4;  however,  we  would  like  to  understand  the 
hinctional  relationship  between  associativity  and  performance.  To  do  this  we  will  use  the 
following  simplified  model. 

Assume  that  a  load  of  size  L  is  chosen  at  random  from  the  total  address  space.  We 
compute  the  performance  of  the  load  by  inserting  the  load  items  into  the  cache.  Some  slots 
will  overflow,  in  which  case  the  extra  entries  remain  in  main  memory.  We  assume  that 
the  program  references  the  load  items  uniformly  at  random,  so  the  pt'rfonnaucc  depends 
on  the  amount  of  cache  overflow.  (This  a.ssumi)tion  makes  the  an;dysis  independent  of  the 
replacement  algorithm,  since  “uniformly  at  nandom”  ignores  the  gain  of  moving  entries  into 
the  cache  after  they  are  referenced.  The  locality  of  program  reference  is  modelled  instead 
by  a  choice  of  small  L.) 

Under  these  assumptions  we  obtain  jui  occupancy  problem.  If  /  is  the  index  size  and 
A  the  degree  of  associativity  then  the  corresponding  occupancy  problem  involves  loading 
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L  numbered  objects  into  /  boxes,  tagging  each  extra  object  after  the  Ath  with  an  overflow 
indicator. 

Here  is  a  grammar  describing  this  process  for  A  ^  2: 

S  D2pS  I  € 

D2  x°Di  I  f 

Bo  ->  ox°Bo  I  £ 

The  p’s  separate  boxes,  the  x’s  are  the  objects,  and  a  prefix  o  indicates  that  an  object  has 
overflowed.  The  grammar’s  <H]uations  can  be  solved: 


Do  =  e*’* 

„  c®*  -  1 

Di  = - +  1 


=  _ - 5+1+1 

1 

S  = 


(3.2) 


1  -  B2P 

From  this  construction  it  is  clear  how  the  associativity  affects  the  generating  fiinction: 

1)  One  way. 

c®*  1 

(3.3) 


-  i  +  1 

o  o 


2)  Two  way. 


B2  =  -5- - «+i+l 

o*  o 


(3.4) 


3)  A  way. 

goz  1 

o{A-  1)!  ~  o^{A  -  2)!  (A-l)!  (A  -  2)!  ^ 

But  it  is  not  clear  that  we  can  obtain  any  meaningful  information  from  these  formulas. 
S  is  a  multivariate  function  of  p,  o,  and  x.  The  coefficient  of  p^  ^  in  S  is  a  generating 
function  in  o  repre.senting  the  distribution  of  overflow  for  load  L  in  7  boxes.  We  can  gain 
information  abotit  overflow  by  differentiating  with  respect  to  o  first  before  we  extract  the 
coefficient  of  p^ 


Average  overflow  = 


o) 


1 


(3.6) 
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The  equation  is  divided  by  all  possible  loadings  of  objects  into  boxes  (/^)  in  order  to  obtain 
L  probability  distribution.  This  is  also  accomplished  with  the  substitution  x  y//* 

Average  overflow  =  -^S[pyyl lyo) 

When  0=1,  the  expression  for  Da^vIUo)  collapses  nicely: 

i?A(y//,l)=e»^^  (3.8) 

The  complexity  lies  in 


£cx(y//,o)  = 


oVv//y//-^o>i-Xe°v/^  ^  (y//)^-»  ^  „  (y//)^-»  ^  ^  _ 


o»(A-l)!  o*(A-2) 


Putting  this  together, 


Average  overflow  =  « 


,  MI)*-'  ,  ;(»//)'■-’  ,  I  V 
+  -frri)r+^A-2)i  ^ 


(3.10) 


thus  for  A  =  2  we  expect  the  overflow  to  be 

L  -  2/  +  L(1  -  1//)^"*  +  2/(1  -  1//)^ .  (3.1 

A  similar  calculation,  using  the  second  derivative  with  respect  to  o,  yields  the  variance. 


For  comparison  of  different  A  a  realistic  standard  is  the  total  volume  V  =  A/.  We 
assume  that  the  load  is  some  proportion  of  this  volume  L  =  aV^.  As  a  — ♦  oo,  L  — V  overflow 
is  expected,  regardless  of  the  <issociativity.  For  smaller  a  the  associativity  plays  a  more 
important  role.  To  examine  this  region  we  let  7  -♦  oo  and  approximate  (1  -  1//)^  with 

M  -aA. 


Average  overflow  =  L  -  V  +  /c 


-{S?9*15S- •')•<■(« 

(3.12) 


-L  y  +  /e  +  (^_2)!  ^  ^ 
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Two  cases  arc  of  interest.  Wlicn  q  0,  the  overflow  is  exponentially  related  to  the 
associativity: 


(  ( 

Average  overflow  ~V  I  a  -  1  -f  ^  I 

where  the  error  is  constrained  by 

1  <  E(a)  < 


Ae°''  -  aAe°''  + 


^  .  2(aA)^^^ 


(A+1)!  (A +  2)! 


1 


(1-ap- 


•••)) 

(3.13) 

(3.14) 


For  small  a  the  most  sensitive  term  in  equation  (3.13)  is  so  the  percentage  overflow 

falls  off  dramatically  with  the  associativity. 

A  second  case  of  interest  is  when  a  =  1,  corresponding  to  a  cache  pressed  nearly  to  its 
limit: 


Average  overflow 


-aA 


(A  -  I)(«A) 


A-1 


(A-2)(aA) 


A-3 


A  V  (A-1)!  (A -2)! 

fr/  .  /.  X  -04 /(oA)^-2  (aA)^-» 

=  K(a-l  +  (l-a)e  +  , 


.o)) 


+  1 


+  e 


-qA 


(aA) 


(A-1)! 


—) 

l)!j 


=  Ve 


AA-1 


(A-1)! 

(3.15) 

Here  the  percentage  overflow  flUls  off  inversely  with  the  s(iuare  root  of  the  associativity. 


Depending  on  one’s  temperament,  there  are  two  ways  of  looking  at  this  last  result. 
Remember  that  the  case  a  =  1  causes  a  fully  associative  cache  to  work  bcautifiilly,  with 
no  overflow  and  full  memory  usage.  The  above  result  gives  encouraging  news:  a  cheaper, 
four  way  asso<*iativo  cache  is  worse  by  only  20%  overflow.  However,  the  restdt  iilso  gives 
discouraging  lu'ws:  when  the  load  factor  nr  nears  1  increasing  the  degree  of  associativity  is 
not  particularly  h(*lj)ful.  It  is  only  when  a  is  small  and  perfornuincc  good  anyway  that  the 
degree  of  associativity  h«as  an  impressive  cfiect  on  the  amount  of  overflow. 


Let  us  pause  for  a  moment  to  compare  the  augmented  grammar  approach  with  the 
traditional  analysis  of  this  problem.  Normally  we  would  assume  that  the  number  of  boxes 
is  large  so  that  wo  could  approximate  the  behavior  of  an  iiidi\id  ial  box  with  the  Poisson 
distribution.  (In  fact,  fragments  of  the  Poisson  distribution  appear  in  the  above  formtilas.) 
Then  we  would  sum  the  expected  overflow  from  each  box  to  obtain  an  average  overflow 
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(a  technique  that  wouldn’t  »vork  for  the  variance).  The  chief  advantage  of  the  augmented 
grammar  approach  is  that  it  leads  gracefully  to  a  multivariate  generating  function  that 
contains  all  the  important  information^  without  approximations.  Approximations  can  come 
later  in  the  process  of  understiuiding  the  asymptotic  features  of  the  distribution. 

3.2  Binary  Tree  Search 

A  binary  tree  is  a  data  structure  used  for  the  storage  and  retrieval  of  information  based 
on  keys  associated  with  entries  in  the  tree.  The  data  structure  is  well  suited  for  applications 
where: 

1)  There  are  roughly  the  same  number  of  insertions  as  retrievals. 

2)  It  is  difficult  to  predict  how  large  the  data  structure  will  grow. 

3)  Average  performance  is  more  important  than  worse  case  behavior. 

4)  The  data  structure  resides  in  main  memory. 

5)  It  is  helpful  to  know  the  order  of  keys. 

If  any  of  these  criteria  are  not  met  then  there  arc  better  options:  when  retrievals  greatly 
exceed  insertions  then  a  balanced  tree  scheme  is  preferable.  Hashing  is  used  in  cases  where 
tlic  table  size  is  fixed  in  advance,  and  the  order  of  keys  is  irrelevant.  When  the  data  size 
exceeds  main  memory  it  is  often  better  to  use  a  structure  adapted  to  the  computer’s  page 
size,  such  as  B-trees.  Nevertheless  there  is  a  distinct  domain  where  binary  trees  axe  the 
best  known  method  of  storage. 

This  section  proposes  a  modification  that  is  intermediate  between  balanced  and  un¬ 
balanced  tree  searching.  Rather  than  completely  balancing  the  tree,  it  only  improves  the 
balance  of  the  data  structure;  wo  will  call  it  diminished  tree  searching.  The  domain  of  use¬ 
fulness  for  diminished  tree  search  overlaps  primarily  with  the  ordinary  tree  search  domain 
described  above;  and,  as  wc  will  see  shortly  in  the  anfJysis,  it  outperforms  ordinary  tree 
search  on  the  larger  problems. 

3«2«1  Diminished  Trees 

In  ordinary  binary  tree  search  the  tree  nodes  contain  a  left  link,  a  right  link  and  a  key; 
and  the  central  loop  of  the  search  routine  looks  something  like  this; 

while  true  do 

if  t^.key  >  sought  then 

if  t].lcft  ^  nil  then  t  t^.left  else  goto  missing 
else  if  t].key  <  sought  then 

if  t].right  ^  nil  then  t  t^.right  else  goto  missing 
else  goto  found 
missing:  ... 
found:  ... 
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Notice  that  there  are  two  exits  from  tlie  loop,  one  for  a  successful  search,  and  one  for  the 
unsuccessful  pt)ssihility  that  the  search  (‘iicounters  a  nil  j)ointer.  Since  the  loop  will  usually 
be  execut  ed  ca  logaritlunic  nuniber  of  times  it  represents  tin*  larj^est  ^asymptotic  contribution 
to  the  nnining  tinu  of  tlie  algorithm.  The  (wits  require  only  a  constant  amount  of  additional 
computation  and  are  therefore  less  important  to  the  overall  performance  of  the  algorithm. 

Recognizing  this  difference  in  cost,  the  modifications  necessary  for  diminishing  the 
searcli  path  are  made  entirely  wnthin  the  exit  code,  leaving  the  inner  loop  of  the  search  in 
its  swift  untouchcil  form.  To  do  this  we  introduce  dummy  nodes  at  the  leaves  of  the  tree: 


Tree  Node 

Accumulator 


Figure  3.3  A  Dummy  Node  and  an  Accumulator 

Due  to  the  infinitely  large  key  at  a  dummy  node,  the  search  loop  will  always  turn  left, 
encounter  a  nil  pointer,  and  take  the  “unsucccssfur  loop  exit,  at  which  point  new  code 
will  pick  up  tlu'  right  pointer  and  search  the  accumulator  list  for  the  desired  key.  The 
accumulator  is  maintained  as  an  ordered  list,  so  if  the  sought  key  is  missing  and  must  be 
inserted  then  shifting  may  be  necessary  to  keep  the  keys  ordered.  The  <aerumuLator  also 
has  a  fixed  odd  size.  2/  +  1.  If  the  arriving  key  fills  the  accumulator  then  the  list  is  split,  the 
median  key  is  placed  in  a  new  tree  node,  and  two  caccumulators  are  created  with  t  elements 
each.  This  is  depicted  in  Figure  3.4.  The  algorithmic  details  can  be  found  in  Appendix  B. 
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Figure  3.4  An  Accumulator  Splits 

The  accumulators  allow  the  tree  to  grow  in  a  (lelayinl  fasliioti.  Those  keys  appearing 
in  tree  nodes  are  chosen  after  +  1  keys  arc  examined.  This  extra  enlightenment  means 
that  tree  nodes  will  more  evenly  split  future  insertions  into  their  stibtrccs  and  so  diminish 
the  path  length  of  the  whole  tree.  However,  we  need  to  study  this  change  carefully  to 
understand  when  path  length  improvements  exceed  additional  accumulator  costs  and  to 
pick  the  best  value  of  the  parameter  f. 

3.2.2  Analysis  of  Diminished  Tree  Searching 

For  the  analysis  it  is  convenient  to  introduce  the  notion  of  <in  ordi»re<l  hotap,  iin  object 
that  several  authors  have  found  usefid  in  studying  binary  trees  [Burge  1972]  [Franqon  1976] 
[Viennot  1976].  A  heap  is  a  binary  tree  with  the  labels  1  to  n  assigned  to  the  nodes  in 
such  a  way  that  each  node  has  a  smaller  label  than  all  of  its  dcsccndents.  An  example  is 
pictured  in  Figmrc  3.5. 
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Figure  3.5  A  Heap 

Search  trees  and  heaps  arc  related  as  follows.  We  take  a  permutation  of  1  to  n  and 
insert  it  into  an  ordinary  binary  search  tree.  At  the  same  time  we  create  a  heap  with  an 
identical  shape  as  the  search  tree,  but  we  use  the  order  of  filling  the  search  tree  to  label 
the  nodes  of  the  heap.  Figure  3.G  shows  the  permutation  and  search  tree  corresponding  to 
the  heap  in  Figure  3.5. 


2413 


Figure  3.6  The  Companion  Search  Tree  and  Permutation  for  Figure  3.5 

Given  a  heap  we  can  recover  the  associated  permutation  by  walking  the  heap  in  infix 
order.  If  the  ith  node  in  infix  order  Inis  label  j  tlien  t  is  inserted  into  position  j  of  the 
permutation.  Tliere  is  thus  a  one  to  one  correspondence  between  heaps  and  permutations 
that  will  make  hea])s  espetdally  tiseful  in  analysis. 


Lvt  us  begin  our  analysis  by  using  heaps  to  study  ordinary  binary  trees.  The  grammar 
for  a  heap  uses  the  box  operator  to  insure  that  every  node's  label  is  smaller  than  those  of 
its  descendants: 

e. 


S  x°lShlSh 


(3.16) 
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And,  not  surprisingly,  the  soltition  of  this  grammar  indicates  that  there  arc  n!  heaps  of 


sire  n: 


s^js^  +  i 

-  ^ 

1  - 1 


(3.17) 


Of  greater  interest  is  tlie  path  length  of  a  heap.  For  any  rooted  tree  the  (internr’)  path 
length  is  defined  to  be  the  sum  of  the  distances  from  each  node  to  the  root.  In  the  ease  of 
heaps  the  path  length  is  the  total  number  of  comparisons  used  to  build  the  corresponding 
search  tree.  Dividing  by  n,  the  size  of  the  tree,  and  adding  one  for  the  last  comparison, 
gives  the  average  number  of  comparisons  for  a  successfid  search. 


To  aiialyze  path  length  we  add  an  index  t  for  depth: 


Si  — ♦  q'x^lSi+ihlSi+ih  c. 


(3.18) 


Each  node  i  has  been  prefaced  by  a  string  of  q'a  recording  its  depth  in  the  tree. 
Converting  this  to  an  integral  cejuation. 


Si 


+  1. 


(3.19) 


and  applying  the  Q  operator  (Qf(x)  =  f{qx))  yields  the  the  same  equation  as  for 


+  1 


QSi  =  q'Q  j 

=  9’^y‘(g5<+i)*  +  i. 


(3.20) 


This  suggests  the  conjeettire  QSi  =■  Si  which  agrees  with  our  intuition:  any  subtree  with 
root  at  depth  i  can  be  moved  to  depth  t  +  1  by  prefacing  each  node  x  with  an  extra  q. 

For  the  whole  tree  we  obtain  the  equation 


S'o  =  (f?5o)» 


(3.21) 


which  does  not  seem  to  have  a  simple  sohition.  With  the  chain  rule  of  differentiation, 
however,  we  can  extract  useful  information  from  the  equation.  To  obtain  the  mean  path 
length,  we  differentiate  with  respect  to  seeking  a  generating  function  in  x  alone: 

(3.22) 


r(*)  =  ^0(1,7) 


The  coefficient  of  i*  in  T{x),  treated  as  an  ordinary  generating  function  in  i,  will  be  the 
average  path  length  of  heaps  of  size  1.  (Ordinarily  all  generating  functions  in  the  labelled 
terminal  x  are  treated  as  exponential.  Li  this  situation  the  t!  is  the  normalizing  factor  for 
the  probability  distribution  ui  path  length.) 
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DifTcrentiatinj;  both  si(li\^  of  (3.21)  with  respect  to  q  and  using  the  cluiiii  rule  reveals 
an  ordinary  difTorential  equation  for 


^ 

dq  Ox 


=  2So{qr,q) 


9=1 


r  = 


1  - 1 


1 


(i-i)r-2r(i)  = 

With  integrating  factor  (1  —  i)“*, 

T(i)  =  (l-i)^>G(x) 

«■'*)  = 

G(x)  =  -21n(l  -  i)  -  2i 
ln(l  -  x)  +x 


r(i)  =  -2- 


c} 

-So(x,q)  +  j- 
=  ^7  J  9=1 

(3.23) 

j + ’•(') 

(3.24) 

2x 

(3.25) 

(l-x)2- 

(3.2G) 


and  we  can  recover  the  cocfRcient  of  x'  u.sing  a  family  of  identities,  due  to  Zave  [Zave  1976), 
for  powers  of  ln(l  -  i)  over  powers  of  (1  —  i).  In  this  case, 

-  l)(n+  1)^",  (3.27) 

SO  the  expected  path  length  is 

2{/f„+,  -  l){n  -  1) -2n,  (3.28) 

and  the  average  successful  search  will  require 

2/y„-3  +  2^  (3.29) 

comparisons. 


The  intent  of  this  section  was  to  study  diniinislied  searching  and  yet  so  far  wo  licivc 
only  managed  to  compute  the  comparisons  for  ordinary  trix?  socarch.  However,  the  steps  of 
the  above  analysis  will  provide  a  model  for  our  analysis  of  diminished  searching.  Recall 
that  wc: 

1)  Established  a  correspondence  betwct'ii  the  data  structure  of  interest  (search  trees) 
and  another  structure  (heaps)  tlnat  logged  tlie  arrival  of  elements  of  the  first  structure. 

2)  Developc<l  a  labelled  grammar  and  converted  it  to  a  differential  c'qnation  in  two 
variables. 
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3)  Ri'inuvrd  one  variable  (rising  the  Q  operator,  and  the  rhairi  rule),  and  solval  the 
reniaininj;  dilferential  expiation  for  a  generating'  function. 

4)  Recovered  tlie  coeflirients  of  the  generating  ftinction. 

These  step.*?  will  be  repeated  in  the  analysis  of  Diminished  Tree  Search,  but  first  let  us 
review  some  mat  hematics  that  will  prove  usehd  in  the  analysis  that  follows. 

In  the  fourth  step  we  made  use  of  an  identity  of  Zavc.  Here  is  the  most  general  form 
of  the  identity: 


(-ln(l-r))"  (rr  IJ  fj{n)  _  +  Aj* 

whore  the  angle  hracketctl  superscripts  indicate  a  truncated  Ricinaim  sums, 

and  the  polynomials  Pn  arc  ndated  to  the  Bell  polynomials 

Pfii^i^  •  •  •  »  ^n)  ~  (“■  j  ~2S3,  .  .  .  ,  —(n  1)!  <Sn)  • 

We  will  make  heavy  use  of  two  subcases: 


(1 


=  E((»»  O'  -  H-)’  -  ('C,  -  0  ^  ■ 


in  order  to  recover  the  coefficients  of  generating  functions. 
In  the  third  step  we  solved  a  differential  eipiation, 

2x 


(i-x)r'-2r  = 


(3.30) 

(3.31) 

(3.32) 

(3.33) 

(3.34) 

(3.35) 


by  using  the  integrating  factor  (I  —  x)  Such  a  clean  integrating  factor  was  possible 
because  of  the  (1  -  x)  factor  prefixing  T'.  In  genend,  wc  ran  solve  higher  order  equations 
like 

a{l  -  x)5r'  +  6{1  -  xf'r  t  c(l  -  x)T'  +  liT  =  0  (3.30) 

cis  long  as  the  power  of  the  prefixing  factor  matches  the  differentiation  of  T.  First  wc  use 
tlie  change  of  variables  t;  =  1  -  x,  and  then  we  replace  differentiation  with  the  operator 
0  -  so  a  term  like  can  be  cxpress(jd  with  a  failing  factorial  of  the  operator: 

Equation  (3.3G)  above  becomes 

b6t?^~ct>  +  d)r  =  0. 


(3.37) 
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li)  gc-ucral  wc  will  have  a  polyTioniial  of  the  operator  applied  to  T: 


P{^})T=0.  (3.38) 

Suppose  wc  have  a  factorization  of  this  polynomial  and  r  is  one  of  roots.  The  contribution 
of  a  single  root  can  be  found  using  for  .an  integr.ating  factor: 


(.?-r)G  =  0 

G  =  v'H 

=  0 

dJl  ^  ^  (3.39) 

dv 

H  =  k 

G  =  kv\ 

where  fc  is  a  const.ant  determined  from  initi.al  conditions.  If  the  equ.ation  is  inhomogeneous, 
with  a  right  hand  side  of  e”,  s  ^  r,  then  the  solution  is  still  straightforw.ard; 


(d  -  r)G  =  w* 

G  =  v^H 


OH 

dv 


—  u* 


1 


11  = 

G  = 


k  + 


fco'  + 


(d-r) 


(s  -  r)  ■ 


(3.40) 


By  combining  these  two  solutions,  wc  can  see  the  effect  of  a  simple  root  in  a  higher  degree 
polynomial.  Let  the  polynomial  have  distinct  roots: 


P{d)  =  (t?  -  ri)(d  -  rj) . . .  (t?  -  r„) .  (3.41) 

Then  P{d)T  =  0  can  be  solved  by  setting  G  =  (t?  -  r2)(d  -  rs) . . .  (t?  -  r„)r.  The  equation 
becomes 

(t?  -  ri)G  =  0  (3.42) 

with  solution  G  =  fcv'’*.  Now  the  remainder  of  the  equation  is  inhomogeneous, 


r2)(d-ri)...  {d-rn)T  =  kv^\ 


(3.43) 


but  wo  can  continue  to  strip  factors  from  tin*  beginning  of  the  equation  until 
obt.ain 


T  = 


R(ri) 


WC  event  iinlly 


(3.44) 


where  7Z(i?)  =  P(d)f(d  -  rj),  .and  the  other  roots  contribute  term.s  tli.at  .arc  not  shown  in 
equation  (3.44). 
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The  solution  of  an  inhomogeneous  equation  with  a  right  hand  side  of  w'  is  slightly 
more  complicated: 

(t>  -  r)G  =  o' 

G  =  v'H 
v'Off  =  v' 

dH  _  _J  (3'*5) 

dv 

H  =  lav  k 
G  =  v'  In  w  +  fcw” . 

So  we  will  also  need  to  be  able  to  deal  with  logarithms  on  the  right  band  side: 

(t>  -  r)G  =  w*  In  w 
G  =  v'H 
v'^H  =  w*  In  w 


_  Jlll_  +  jfe 

(,-r)  (.-r)>  + 

_  oMnv  _  t>* 

^-(s-r)  * 

Once  again  we  can  combine  these  two  solutions  to  see  the  effect  of  a  single  root  that  is 
equal  to  the  power  of  the  inhomogeneous  right  hand  side.  We  assume  that  the  other  roots 
are  distinct: 

P(t?)T  =  t>'‘ 

G=(t>-ra)(t>-rj)...(t)-rn)r  (3.47) 

(t?  -  ri)G  =  w" 

G  =  v'*  Inv  +  fco*"* . 

Repeating  this  technique  for  the  other  roots  modifies  the  constants  on  the  contribution  of 
the  first  root  until  finally, 

u^'lnw  ,,  ,  k  ,  (%Aa\ 

2*  — i'  -u'* +••• ,  (3.48) 

-^(>•1)  {R(rj))  ^('■i) 


where  again,  R^d)  —  P[0)l[f9  -  ri)  and  fc  is  a  constant  determined  by  initial  conditions. 
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With  the  niatheniatiral  prcliniiiiarios  completed  we  can  begin  the  analysis  of  Dimin¬ 
ished  Tree  Search.  The  data  structure  is  encoded  with  the  usual  /  <and  h  charactcis  for  the 
tree  portion  of  the  structure.  Tlie  x  characters  are  used  for  valid  k('ys  present  in  the  internal 
tree  nodes,  and  t/'s  are  used  for  the  dummy  -|-oo  keys  in  the  leaves.  The  accumulator  lists 
appear  as  contiguous  blocks  of  x's,  enclosed  w'ith  6's  and  e’s.  Ignoring  the  labelling  of  the 
x’s  for  the  moment,  the  following  grammar  encodes  the  shape  of  a  diminished  tree: 

5  xlShlSh  I  ylhlAh 

1  1  (3.49) 

A  “♦  bxxe  I  bxxxe  j  bxxxxe . 

In  this  case  tlie  accumulator  parameter  is  i  =  2,  so  the  acctimulator  lists  will  range  in  size 
from  2  to  4.  Wc  assume  that  the  tree  always  has  at  least  t  keys. 

Following  the  analy.^is  completed  already  for  ordinary  tree  search,  the  I’s  are  labelled 
with  the  order  they  were  first  inserted,  rather  than  using  the  actual  keys.  The  labels  arc 
constrained  by  the  construction  process  for  the  tree.  Since  a  tree  node  results  from  the 
splitting  of  an  accumulator  list  of  size  2i  -h  1,  we  expect  that  the  smallest  2/  +  1  labels  of 
the  subtree  (representing  the  first  keys  inserted)  will  be  divided  evamly:  t  will  appear  in 
the  right  subtree,  t  will  appear  in  tlie  left  subtree,  and  one  will  appear  as  the  key  of  the 
tree  node.  The  following  partial  order  expresses  the  constraint  on  labels  of  a  subtree: 


Figure  3.7  The  Distribution  of  Labels  in  Subtrees 


Denoting  the  partial  order  by  B  we  can  augment  the  grammar  so  that  it  specifies  a  labelled 
formal  language: 

S  I  ylhlAh 

I  I  (3.50) 

A  bxxe  I bxxxe  j  bxxxxe . 

The  partial  order  B  is  well  separated:  in  Figure  3.7  g,  /,  a,  m,  and  n  are  active,  wliile 
h  and  o  arc  boundary  elements.  To  convert  the  grammar  to  an  equation  we  multiply  the 
number  of  linear  embeddings  of  S,  +  1),  by  a  repeated  intcgriil: 

r(2t4i) 


. w--. 


+ 


+  x 


3t 


(3.51) 
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For  the  moment  we  liavc  dropped  all  of  the  terminal  characters  from  this  equation.  A 
superscript  in  parentheses  indicates  repeated  diircrcntiation  unless  it  appears  on  an  integral 
sign,  in  which  case  it  signifies  repeated  integration.  DilFcr^’iitiating  both  sides  of  equation 
(3.51)  yields: 

5(31+1)  =  +  ^3  52) 

with  solution  5  =  1/(1  -  x),  indicating,  as  we  would  expect,  that  there  are  n!  trees  with  n 
keys,  one  for  each  permutation.  The  initial  conditions  that  were  present  in  equation  (3.51) 
but  disappeared  with  the  differentiation  force  us  to  modify  the  solution: 

5  =  7^.  (3.53) 

The  first  f  -  1  terms  of  the  scries  are  now  absent  due  to  the  fact  that  the  grammar  cannot 
derive  a  diminished  tree  with  less  than  t  keys.  In  practice,  of  course,  the  algorithm  starts 
with  an  empty  tree  with  one  accumulator  list  that  absorbs  the  first  few  keys.  Once  this 
list  contains  t  or  more  keys,  we  can  claim  that  all  accumulator  lists  in  the  tree  contain 
between  t  and  2t  keys,  so  that  the  above  grammar  is  valid.  We  could  modify  the  grammar 
to  account  for  the  initial  anomaly,  but  it  is  easier  to  use  the  unmodified  equation  with  the 
qualification  that  n  >  For  n  <  2f  the  algorithm  behaves  like  an  ordinary  insertion  sort. 


Most  qiiantities  of  interest  in  the  analysis  of  diminished  tree  search  arc  obtained  from 
suitable  variations  of  the  grammar  proposed  above.  Suppose,  for  example,  wo  are  interested 
in  the  number  of  nodes  in  a  data  structure.  Ordinary  trees  contain  one  node  for  each  key, 
but  diminished  trees  have  dummy  nodes  with  no  key  at  all,  and  accumulator  lists  with 
several  keys.  The  number  of  nodes  corresponds  to  the  number  of  allocations  during  the 
running  of  the  algorithm— an  operation  that  is  often  expensive  in  liigh  level  programming 
languages. 

To  study  the  number  of  allocations  an  c  is  added  to  the  grammar  at  each  point  that 
a  new  node  is  used: 

S  — ►  [S]<^^|a]^^(/!;A)^^^[Tnno]^  |  CylHlcAH 

I  I  I3.04j 

A  hxxe  I  6xxxe  |  bxxxxe . 

This  converts  to  the  equation 

5(3.+ 1)  ^  (f  +  i)c(5(*>)’  (3.55) 

with  initial  values 

S  =  c^x*  +  +  c^x^^^  +  •  •  •  4-  c^x^*  -I- . .  ♦  (3.56) 

which  does  not  appear  to  have  a  nice,  closed-form  solution.  Nevertheless,  we  can  simplify 
the  problem  by  letting 

c)5(x,c 
dc 


(3.57) 
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The  coefricieiit  of  x'  in  T{x)  is  the  average  nnnher  of  nodes  in  a  tree  containing  t  keys. 


DifTorentiating  equation  (3.55)  with  respect  to  c  gives  an  equation  for  T: 

J.(ii4i)^  +  +  (3.58) 

(1  - 1)^‘  <  =  i^±il: .  (3.61) 

H  1  —  X 


Here  S^^^|c=i  is  obtained  by  differentiating  the  solution  S  =  1/(1  ~  x)  of  equation  (3.52) 
above. 

The  substitutions  v  =  l-x,  t?  =  u^  transform  the  last  equation  to  a  form  we’ve 
explored  already, 

V 

The  polynomial  of  the  operator  t?  depends  on  the  parameter  t: 

p,(,J)  =  ,?2i±i  +  (_i)‘2(2i  +  l)i±l.t>t.  (3.63) 

This  polynomial  appears  in  the  analysis  of  median  of  n  qiiicksort  (sec  [Sedgewick  1975]), 
and  will  appear  again  in  the  study  of  other  aspects  of  diminished  searching.  It  is  worth 
diverting  now  to  study  the  properties  of  P((t?). 


Since  t?*  can  be  factored  from  equation  (3.C3)  we  know  that  t?  =  0, 1,2,. . .  —  1  are 

all  roots  of  ^(t?),  leaving 


-  tY-i±+  (-l)‘2(2t  +  l)i±i^  . 

There  arc  other  integer  roots.  Whenever  t  is  odd  =  3t  +  2  is  a  root: 

(tf  -  1)111  =  (2t  +  2)i±i 
=  2(2t  +  l)l±l, 

and  1?  =  — 2  is  also  a  root,  following  the  observation: 

(,J-t)l±i=(-2-f)*±i 

=  (-l)i±i(2f  +  2)i±i 
=  (-l)ill2(2t  +  l)i±l. 


(3.64) 


(3.65) 


(3.66) 
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As  t  grows  large,  we  can  express  the  falling  factorials  of  equation  (3.CC)  with  gamma 
functions  and  use  Stirling’s  approximation: 


i\~t  + 1?  + 1)  _  r{— f  “  1) 
r(-2t  +  t?)  ”  r(-2t-2) 

g2t-i»(_2t  +  ,?)-2t+'»-i/2  e5t+2(_2t_2)  -2‘-=-‘/a  ^  ^ 

/  a  (  <1  i  I  \ 

^  +  l)‘+‘  (1  +  0(r ‘)) 


t?  +  2'\*''’*  ,_jt_3-l/2 

t+tJ  ’’ 


<  +  2  y-'+i/i 

2-^-8 


i  +  o(r‘) 
i  +  o(r^) 


(3.67) 


So  the  other  roots  converge  towards  the  points  -2  ±  as  shown  in  Figure  3.8,  where 
the  smooth  curves  sketch  the  discrete  trajectories  of  the  roots  as  they  were  located  with 
MACPYMA’s  root  finding  algorithm. 

The  root  with  the  smallest  real  comp  .ic-nt  makes  the  largest  contribution  to  the  asymp¬ 
totic  growth  of  the  coefficients  of  the  solution.  In  the  Ccise  of  t?  =  —2  is  the  dominant 

root,  so  it  will  prove  useful  later  to  factor  d  +  2  from  the  polynomial: 


(d  -  t)i±i 

d  +  2 


d-l-2 


=  (2<  +  2)(d-0 


«.Ji?-2t-l) 
d  +  2 


(3.68) 


=  ^(-l)>(2t  +  2)i(d  -  i)t±+  (-l)‘+‘ 

3=0 


This  means  that 


P,(d)  =  (t?  -h  2) 


-l)^(2f-h2)i(t?-i) 


(3.69) 


/ 


Figure  3.8  The  Roots  of  Po(<?) 

Several  constants  related  to  the  Pt{'9)  polynomials  will  also  prove  useful.  Let 


(3.7( 


We  wish  to  compute  Rt(-2)  and  R{(-2)  in  order  to  apply  equation  (3.48).  Using  LTIopital 
rule  these  quantities  can  be  expressed  in  terms  of  P«(i?): 


ANALYSIS  OF  DIMINISHED  TREE  SEARCHING 


57 


Since  /’,(i?)  is  the  difference  of  two  falling  factorials  of  t?  (sec  equation  (3.C3))  we  can 
compute  these  constants  once  wo  compute  the  derivative  of  a  falling  factorial: 


=  -  J?  -  i)i 

For  0  =  -2: 


at? 


tf=-i 


a» 

at?2 


ai 


=  (-1)' ((^i  M  - 1)*  -  +  i)(j  + 1)!  • 


Assembling  these  results  gives  the  constants  of  interest: 
i?t(“2)  =  (2i  +  2)!  (//2t+2  ““  i) 

i?;(-2)  =  (Hi,  -  Hl^,  +  hI\)  +  i?,(-2) . 


(3.72) 


(3.73) 


(3.74) 


Returning  to  equation  (3.62)  for  the  number  of  nodes  in  a  diminished  data  structure, 
the  careful  study  of  diffcreiiticd  equations  and  /»(*?)  Kow  bears  fruit: 


p,(t?)r  = 


(2f  + 1)! 

V 


r  = 


i?.(-2) 


(2f  + 1)1 

P.(-l) 


.-I 


+  ••• . 


(3.75) 


The  constant  k  is  our  remaining  concent.  Rec<-Ul  that  k  appears  after  the  first  root,  a  +  2, 
is  factored  front  leaviitg  the  cqtiation 


=  (3.76) 

We  can  use  the  initial  conditions  to  compute  /?t(t?)rivi.-i  and  thereby  find  a  value  for  k. 
The  first  few  terms  of  S, 


S  =  c’l*  +  c*!*"*^*  4-  ...c’x’*  +  ...  , 


(3.77) 
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give  tho  first  few  ti-rins  of  T: 


r=  2i‘f2x'  +  ‘ +  +  . 


So  initially  we  know: 


0  <  j  i 
t<j<2t  ■ 


Or,  changing  variables  and  using  the  j?  operator 


0 

(-l)^2j! 


0<j<t 
t<j<t  ' 


By  rewriting  Rt['3)  slightly  we  have 


i-0 


so  that 


Solving  for  k\ 


gives  a  dominant  term  of 


=  X](2i  +  2)i2(2t-i)! 

j  =  0 


1 

(2f-h2-»(2^  +  l.-j) 


=  2(2, +  2),  (^.^) 
=  2(2t+l)!. 


fc  =  iZ,(t>)T|„=i  +  (2t+l)! 
=  3(2t  +  l)!, 


T  = 


1 


2t  +  2  ^2t-i-a  -  /ft +1 
which  expands  with  the  binomial  theorem: 

3  1 


v-*  +  . 


(i")  T  = 


2<  +  2  Hit+j  ~  fft+i 


(n  +  1)  +  o(n) , 


So  the  number  of  allocations  necessary  to  build  a  diminished  tree  is  proportional 
number  of  keys  and  when  t  goes  to  infinity  tho  iUlocations  fall  off  inversely  with  t. 


(3.78) 

(3.79) 

(3.80) 

(3.81) 

(3.82) 

(3.83) 

(3.84) 

(3.85) 
to  the 
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Tho  error  term  on  oqtiation  (3.85)  is  necessarily  weak,  since  tliis  estimate  depends 
on  the  other  roots  of  Pt(d)  ainl  sotne  of  tliese  roots  are  inigratint;  (as  t  oo)  towards  points 
with  -2  real  part.  In  practice,  however,  the  real  part  of  these  migrating  roots  is  still  positive 
wluni  t  —  0,  as  shown  in  Fig\ire  3.8,  and  further  analysis  will  suggest  tha:  we  would  never 
w\ant  <an  accumulator  as  large  as  19,  so  we  may  safely  use  an  error  estimate  of  0(1)  in  the 
above  formula. 

Another  quantity  of  interest,  the  total  memory  tisage,  is  closely  related  to  the  total 
number  of  nodes  in  the  data  stnictiire.  There  are  two  approaches  to  the  amdysis.  The 
more  systematic  approach  uses  a  modification  of  the  above  grtammar: 


S  -*  \B\Tn^x^a]l  S>jgh]hlS^mno]h  I  m^ylhlm*Ah 

A  —*  here  | bxxxc  | bxxxxe . 


(3.8C) 


The  amount  of  memory  constimcd  appears  in  the  exponents  of  the  m*s:  3  spaces  for  a  tree 
node  and  4  spaces  for  an  accumulator.  In  general,  an  accmnulator  takes  2t  spaces,  althotigh 
it  is  sometimes  easier  to  program  with  2f  -f  1  spaces  so  that  the  accumulator  can  gracefully 
absorb  (2t  1)  keys  before  splitting. 

The  technique  used  here  deserves  highlighting.  Wc  have  see<le<l  the  original  grammar 
with  a  dummy  variable,  m,  raised  to  the  powvr  of  a  (piantity  of  interest,  in  this  case  the 
amount  of  memory  used.  We  will  repeatedly  use  this  technique  to  study  other  quantities 
of  interest,  such  as  comparisons  and  memory  probes. 


The  conversion  and  solution  of  grammar  (3.8G)  i.s  almost  identical  to  the  node  counting 
grammar  just  solved,  so  it  is  omitted.  The  average  anunuit  of  memory  used  to  store  n  keys 
turns  out  to  be: 

- !__(„  +  l)  +  o(n).  (3.87) 

{  -f  1  /22t+2  “  1 


Comparing  this  with  tlie  3n  memory  required  for  ordinary  tree  search,  wc  sec  that  the 
memory  usage  increases  to  y  n  for  i  =  1,  but  then  shews  improvement  for  t  >  1,  converging 
to  i^n  as  t  oo,  a  savings  of  52% ! 


An  alternate  approach  to  this  analysis  relates  nodes  and  memory  usage,  so  that  only 
one  of  the  hast  two  comptitations  is  mxessary.  A  diminished  tree  will  talways  have  an 
odd  number  of  tree  nodes,  call  this  2j  -  1,  of  which  j  arc  dumniy  nodes  and  j  -  I  are 
internal  nodes.  The  j  dummy  nodes  have  j  accumulators  as  right  descendants.  Wc  can 
compute  the  total  number  of  nodes,  nodes  =  Zj  ~  1,  and  the  total  amount  of  memory, 
memory  -  (6  +  2t)j  -  3,  and  then  relate  the  two  quantities: 


memory  =  ^  —  (nodes  +  1)  —  3.  (3.88) 

o 

Since  this  is  a  linear  relationship,  the  mean  and  variance  of  these  two  quantities  are  similarly 
related. 


/ 

/ 
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Tho  next  analysis  presents  fresli  (liiTirnltirs,  but  it  cilso  answers  a  critical  question 
about  tlic  nsefubu'ss  (>f  the  new'  data  structure:  on  the  average,  how  many  comparisons  arc 
required  to  fnul  a  k('y  in  a  diminished  tr(T?  This  involvc's  studying  path  length  in  much 
the  same  way  we  studied  paths  in  ordinary  trees,  using  an  imlex  on  the  nonterminals  to 
record  their  depth  within  the  tree: 

E4ich  X  in  the  tree  is  prefixed  with  q  raised  to  a  power  equal  to  the  nnmbcr  of  comparisons 
necessary  to  find  the  key  stored  at  that  location.  The  total  number  of  q's  in  a  tree  divided 
by  the  number  of  keys  is  the  average  successful  search  time  for  the  tree. 

The  graniimar  converts  to  an  e<piation, 

'  ‘  '  •'  (3.90) 

+  7“*r,‘)x‘  + 

where  is  a  function  of  x  and  q.  Applying  the  Q  operator  to  this  liquation  wo  find  that 
Sk » 1,  so  by  tile  fixed  point  theorem  of  Chapter  1,  Si  can  be  expressed  as  a  function 
of  itself: 

/2£  + 

Si  =  <i{  )[t  +  l)f 

^  ^  ■'  (3.91) 

+  7‘  +  (‘V)x‘  +  g‘"‘"('e)x‘^«  +  ...  +  ,’‘*(’‘;')x2‘  . 

This  has  a  differential  form 


with  iiutial  solution: 


5i  =  7(  »’)  ‘x'+gC*’)  'x*^‘ +---  +  7("»')  *x’‘  +  '-- .  (3.93) 

To  find  the  mean  of  the  total  path  length  we  let 

.iinl  tin'll  dilforciitiafi'  I'qiintion  (3.92)  with  rospi’ct  to  q  ami  sc’t  7=1. 

r(2..i)  ^  ^21+  ^  +  i)(i,>(0)2  +  26'<‘)(x5‘^‘  +  r<‘)))  .  (3.95) 

Here  we  have  u.sed  the  chain  nile  to  differentiate  Once  q  is  set  to  one  Si  becomes 

^  =  l/{^  ~  replared  using  the  formula: 


5(‘>  = 


(1  -  x)‘ 


(3.96) 
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This  yields 

(1  - x)’‘+ -2{2t+iy^{i - i)‘r<‘>  =  (21 + 1)!  + (2t + 1)!  .  (3.07) 

1  —  X  [l  —  X) 

Aiid  with  the  siibstitvitions  w  =  1  -  x,  and  t?  =  we  find  a  familiar  form  of  differential 
equation  with  a  familiar  polynomial  P»(d): 

Pt[0)T  = 


p,(d)  =  +  {-l)‘2(2t  + 


2(t-f  l)(l-v)' 


(3.98) 


This  time  however  there  is  an  inhomogeneous  term,  ~(2t  4  2)!  with  an  exponent  that 
matches  the  dominant  root  t>  =  -2  of  A(^)>  ^  situation  dealt  with  in  eqtiation  (3.48).  This 
gives  a  leading  term  of 

r  =  -(21  +  2)1  --p^  +  . .  .  (3.99) 

Replacing  v  with  1  -  x  and  applying  Zave's  identity  yields 

(x")  T  =  - - (//„.,  -  l)(n  +  1)  +  o(n) .  (3.100) 

«2t4 a  “  "t  f  1 

Comparing  this  with  the  leading  term  of  etjnation  (3.28),  2[Hn^i  -  l)(n  -  1),  we  see 
that  when  t  =  1,  diminisljcd  searching  already  shows  an  advantage  (12/7  versus  2  in  the 
constant)  over  ordinary  sccirching.  For  large  n  this  is  a  14%  improvement  in  the  number 
of  comparisons  nccess*iry  for  successful  sc*arching.  As  t  grows  large  the  leading  constant 
converges  to  l/lu2,  a  28%  improvement  in  the  number  of  comparisons. 

However,  we  must  not  rejoice  premcaturcly  since  the  next,  linear,  term  in  the  expansion 
can  have  a  dramatic  effect  on  the  usefulness  of  the  algorithm.  Compare 

2NlnN  +  2N  (3.101) 


jNlnN-^aN,  (3.102) 

While  the  st'cond  formula  is  iisyniptotically  14%  smaller  than  the  first,  the  break  even  point 
is  larger  tlian  ii  million.  Fortunate iy  «i»o  break  even  point  for  dimhnshed  searching  is 
considerably  smaller,  but  in  order  to  satisfivctorily  iinalyze  tlu'  algorithm  we  must  compute 
the  constant  in  the  second  term  of  the  expansion. 


To  obtain  this  constant  we  return  to  the  solution  of  the  differential  equation.  After 
+  2  is  factored  from  the  equation  there  is  tan  undetermined  constant,  k,  in  the  remaining 
equation: 

Inv  jfc  (2f-H)! 


I?  nr - /o# 
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Wliru  the  other  roots  are  rcmovtxl,  k  remains  in  tlic  solution; 


-(2<  +  2)! 
7?, (-2) 


+  (2t  +  2)! 


Rt(-2y 


k  1 
7?,(-2)  t;5 


(3.104) 


The  cortstant  k  is  found  by  setting  v  =  1  in  equation  (3.103)  and  computing  /?£(t?)T||,~i 
from  the  inithil  condilions.  B;\sed  on  equation  (3.93)  the  first  few  terms  of  T  are: 


’■ = (('  2 "  (C  r)  - *■■■*  (C'j ')  - 

This  gives  initial  conditions  of 

r<^^(o)  =  {j!((q2j  _  1)  t<j<2t, 

or 

=  {(-l)7j!((>j*)  -  1)  1  <j<2t  ■  (3.107) 

We  can  now  compute 

I 

i?.(«?)n»=i  =  +  2)ii??i^ru=, 

=  ^(21  +  2)i(2t  - 

So  k  is  given  by 

(3.109) 

Then  by  ^apidying  Ziivc's  ide  ntity  to  equation  (3.101),  and  using  the  results  /?t(-2)  and 
/?i(~-2)  computed  already,  wc  gel  the  desired  solution: 


(x”)r  —  j- - ~  l)(n  +  1)  -f- 

il2f  +  2  -  1 

__J _  (  t±i  ,  +  ,  1 

2  2  2  2  'H2tn-Ht^i 


(n  +  1)  +  o(n) . 


(3.110) 
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The  table  below  sunitnarizes  these  constants  for  small  values  of  t.  The  break  even  column 
on  the  ri|;ht  shows  when  an  accumulator  of  size  t  proves  better  than  an  accumulator  of  size 
i  -  1.  For  f  =  1,  the  algorithm  performs  better  than  ordinary  tree  search  once  n  exceeds 
20. 


Half  Accumulator 

Coefficient  of 

Coefficient  of 

Break  even  n 

t 

(n+l) 

t  with  t  -  1 

No  accumulator 

2 

-1 

1 

y  ~  1.714 

-.245 

49 

20 

2 

^  ~  1-622 

37 

-  .212 

1369 

210 

3 

^  ~  1.570 

533 

103060 

- ^  .680 

284089 

4.3  X  10< 

4 

2520  , 

^  1.549 

1627 

3118374 

-  1.178 

2647129 

1.5  X  10* 

Many  qtiantitics  of  interest  can  be  analyzed  in  way  similar  to  the  analysis  just 
completed.  For  example,  perhaps  tlie  most  realistic  estimate  of  the  cost  of  successful 
searching  is  the  mimber  of  memory  references  made  by  the  algorithm.  The  diminished  tree 
algorithm  wastes  3  memory  references  on  the  dummy  nodes  but  then  recoups  some  of  this 
loss  on  the  linear,  one  probe  per  key,  search  of  the  accumulator  lists.  If  we  assume  that  one 
probe  is  necessary  to  find  the  root  of  the  tree,  then  the  appropriately  weighted  grammar 
is: 


lB\q 

,It+3 


H|mno| 


MS, 


Sk^ 

A,  I  bq“’*  "xq"'' '  "xq"''  ’  ''xe 


ylhlAkh 

2k  C5 


(3.111) 


This  time  the  infinite  set  of  equations  is  resol  vimI  with  a  double  application  of  the  Q  operator, 
Sk^i  ^  Q^Sk>  The  average  number  of  probes  for  a  tree  of  size  n  tunis  out  to  be: 


^2t+2  “ 

1 


i 


l)(n  +  l) 

t  +  1 


^2t  +  2  —  Hi  fl 


+  +  2 


H. 


<2> 

2t  :  2 


»<2) 


H2t  \  2  “ 


rn) 


-hl)+o(iV). 

(3.112) 


To  study  comparison.'?  in  unsucces.'«ful  searcliing,  we  insert  v’s  between  each  of  the  nodes 
in  the  accumulator  lists.  Any  new  key  will  land  in  one  of  these  slots,  and  the  exponent  of 
the  q  will  encode  the  number  of  conparisons  used  to  locate  the  correct  slot. 

Sk^\B\xlSk,ihlS,,^i\ymAkh 

Ak  -*  '■’c  |  bq^^^xq'‘^^xq^^^xq^*  ^c  j  bq^^^ xq^^  ^xq'‘^^xq^^*xq^'*'*e . 

(3.113) 
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This  set  of  (Hpiations  is  resolved  with  the  operator  qQ^  and  the  generating  function  has 
coi’fhcicnts 


1 


//jt  ”  //t4l 


(^n4l  1)(R  +  1)  -f 


Hu.7  -  //..I  [-^  +  2  -  2  +  1  -  2  ("  +  !)• 

(3.114) 

Dividing  by  n  +  1  gives  the  average  number  of  compcirisons  in  an  iinsticcessful  search. 
Equation  (3.114)  compares  favorably  with  a  similar  quantity  for  ordinary  tree  search: 


2(/fn4i-l)(n+l). 


(3.115) 


However,  the  most  interesting  quantities  for  unsuccessful  searching  are  the  memory 
reads  and  writes,  especially  in  cases  where  the  accumulator  lists  arc  splitting.  TVeating 
writes  separately  from  reads,  we  obtain  a  grammar  for  writes: 


Sk  ^  lS]xl5fc^  iW5k+,  I  ylhlAkh 

Ak  -*  bw^xw^xwe  I  bw* xw^ xw^ xwc  | 


Most  of  the  w  exponents  correspond  to  moving  segments  of  the  accumulator  lists  and 
writing  the  new  key.  But  when  the  accumulator  is  full  there  is  an  extra  cost  for  creating 
new  tree  nodes  and  copying  half  the  old  accumulator  into  a  new  accumulator.  This  is  11 
in  the  above  formula,  and  will  be  t  +  9  in  general  (t  for  copying  half  the  accumulator,  6 
for  writing  the  fields  of  two  new  dummy  nodes,  and  3  for  rewriting  the  old  dummy  node 
and  its  pointers  so  that  it  contains  the  median  key).  Solving  (3.116)  yields  a  generating 
function  with  coefficients  equal  to  the  average  number  of  writes: 

1  / 1  1  1 

This  is  slightly  worse  than  the  4  writes  required  for  ordinary  tree  search. 

M%.mory  reads  are  set  up  as  follows: 

Sk^lB]xlSk-ihlSk^i\ylhlAkh 

bq^'‘'*xq^’‘^*xq^^^*^  |  6#/** '  '  '^*i<7**^*c  |  (3.118) 

5^2*  ♦6^  0  eC  ( 3^^2*  ^0+Sg 

In  most  cases  the  algorithm  will  use  2fc  +  2  reads  to  reach  the  accumulator,  and  then  it  will 
read  the  entire  accumulator,  cither  to  find  the  proper  location  for  the  new  key,  or  to  move 
the  keys  that  were  not  searched.  When  the  accumulator  is  full,  there  is  extra  reading  (3 
probes  in  equation  (3.118),  and  t  +  1  in  general)  to  remove  the  median  and  the  larger  half 
of  the  accumulator. 
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This  family  of  equations  is  resolved  with  q^Q^,  and  solved  with  the  usual  techniques: 


•ffjt+a  ~  Ht+i 


l)(n  +  l)  + 


This  is  to  be  compared  with 


-Ht+i  ( 


5  -  //,4 1  ^ 


4{i/„+i-l)(n  +  l)  +  (n  +  l), 


(3.119) 

(3.120) 


for  ordinary  tree  search.  To  get  the  average  number  of  memory  reads  we  divide  these 
equations  by  n  +  1. 


Without  knowing  the  exact  costs  of  various  machine  operations  and  the  nature  of  the 
compiler  used  it  is  difiBcult  to  make  a  precise  recommendation  for  the  use  of  diminished 
searching.  The  figures  below  summarize  break  even  points  for  some  of  the  fac^ors  likely  to 
affect  the  problem.  They  a:e  based  entirely  on  memory  roads  and  writes  and  the  assumption 
that  they  both  cost  one  unit  of  time. 


O  s  Ordinary  M  n  s  MeiJian  of  n 


Figure  3.9  Break  Even  Points  and  the  Ratio  of  Successful  and  Unsuccessful  Searching 
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O  =  Ofdinary  M  n  =  Median  of  n 


Figure  3.10  Break  Even  Points  and  the  Cost  of  Node  Allocation 

Of  coiirse  the  exact  choice  of  algorithm  depends  on  a  combination  of  the  equations 
of  this  section  with  appropri.ate  cost  estimates.  Nevertheless,  these  figures  give  ns  a  good 
pcrspt>ctive  on  the  design  space  of  the  problem.  Notice  that  median  of  3  is  completely 
skipped;  it  is  better  to  use  either  median  of  5,  or  ordinary  tree  search.  This  Jump  was 
anticipated  by  equation  (3.87),  showing  that  total  memory  use  increased  for  median  of  3; 
it  appears  that  median  of  5  is  better  for  both  access  time  and  memory  fisc. 

The  designer  should  be  particularly  .sensitive  to  the  way  memory  is  allocated,  since  tliis 
can  significantly  affect  the  choice  of  t.  With  a  simple  stack  allocator  the  best  choice  of  t  is 
usually  2  (median  of  5).  If  a  compiieated  memory  manager  is  used,  then  larger  values  of  t 
will  greatly  reduce  the  number  of  allocations  (see  e<iuation  (3.85)),  decreasing  the  constant 
on  the  linear  term,  and  shifting  the  break  even  points.  In  the  region  of  1000  keys,  a  choice 
of  t  equal  to  3  or  4  becomes  attractive. 

Finally  it  is  worth  noting  two  features  not  shown  in  Figfircs  3.9  and  3.10.  These 
figures  show  break  even  points  between  algorithms— not  points  of  substantial  advantage. 
To  obtain  a  point  where  half  the  asymptotic  improvement  is  realized,  a  good  nde  of  thumb 
is  to  .sciuare  the  break  even  point. 

The  figures  .al.so  ignore  improvements  duo  to  bUick  memory  access.  Many  larger  systems 
fetch  blocks  of  4  8  words  from  main  memory,  and  have  hist  instructions  for  moving  blocks 
of  memory.  Both  of  these  features  hfirmonize  well  with  dimiTiished  tree  searching. 


Chapter  4 


Generation  and  Recognition 


While  the  previous  chapter  used  labelled  grammars  to  analyze  algorithms  that  were, 
at  least  on  the  surface,  unrelated  to  labelled  formal  languages,  this  chapter  is  devoted  to 
algorithms  that  operate  directly  on  the  grammars.  Since  a  grammar  describes  a  family  of 
combinatoried  objects,  it  is  reasonable  to  ask  all  the  standard  qxiestions  about  that  family: 
Can  we  generate  a  member  of  the  family  uniformly  at  random?  Can  we  generate  all 
members  of  the  family?  Except  we  ask  these  questions  in  a  more  general  setting:  Can  we 
build  a  system  to  accept  an  arbitrary  grammar  and  then  generate  all  objects,  or  generate 
objects  uniformly  at  random  from  the  specified  family? 

Such  a  gen er allpurpose  system  is  possible.  It  takes  as  input  a  labelled  grammar,  and 
then  provides  a  number  of  functions  related  to  the  grammar.  These  functions  fall  into  two 
categories.  The  generation  category  includes: 

1)  Computing  the  size  of  a  specified  family  of  combinatorial  objects. 

2)  Selecting  an  element  by  rank  within  such  a  family. 

3)  Generating  an  clement  uniformly  at  random  from  the  family. 

4)  Generating  all  elements  in  the  family. 

The  specification  of  a  family  has  two  parts.  The  grammar  provides  what  might  be 
callcfl  a  shape  description;  it  could,  for  example,  constrain  the  strings  to  represent  permu¬ 
tations  in  cycle  format: 

P  C°bP  I  € 

C  -» x°R  (4.1) 

R-*xR\€. 

The  user  must  also  provide  a  size  dcscriptioa,  that  is,  counts  of  critic<il  characters  in  the 
string.  For  this  grammar  the  user  might  specify  3  6’s  and  6  I’s.  With  both  a  shape  and  a 
size  description  a  general  system  can  count  the  family  members  (there  are  225  permutations 
of  6  elements  with  3  cycles)  or  generate  instances  at  random  (such  as  XiX2X^bxzx^bx4b). 

Hereafter,  we  will  refer  to  those  characters  whose  occurrences  are  of  interest  as  critical 
characters.  In  the  last  example  z  and  6  arc  critical.  The  number  of  critical  characters  is 
the  eJimension  of  the  problem,  aiid  the  counts  of  critical  characters  in  the  size  description 
form  a  characteristic  vector.  By  convention,  the  count  of  the  labelled  character  will  appear 
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in  tlio  first  coinpoiu'nt  of  the  vector.  Returning  again  to  the  above  example,  we  have  a  two 
dimensional  characteristic  vt^rtor  (0,3). 

The  second,  recognition,  category  is,  in  a  loose  sense,  the  niverce  of  the  generation 
category.  It  includes: 

1)  Testing  if  a  string  can  be  derived  with  a  grammar. 

2)  Ranking  a  string  within  a  specified  family. 

For  example,  tester  would  reject  X3X16X26  since  it  cannot  be  derived  with  the  above 
grammar,  while  a  ranker  might  give  xi:r^bx2b  a  rank  of  2  witliin  the  family  of  permutations 
on  three  elements  having  two  cycles.  The  rank  has  no  absolute  meaning;  we  do,  however, 
expcx'l  that  each  member  of  the  family  will  have  a  ditferent  rank,  and  that  the  ranking 
function  of  this  category  is  an  inverse  of  the  selection  function  of  the  preceding  category. 


These  generation  and  recognition  problems  have  received  considerable  attention  in  the 
literature.  There  care  specml  purpose  algorithms  for  Vearious  combi natorical  objects,  and 
general  framew’orks  for  Large  classes  of  problems  [Nijonhuis  1978]  [Wilf  1977  and  1978] 
(Williamson  1976].  The  interest  stems  from  numerous  applications:  Hypotheses  can  be 
tested  with  complete  sccarches.  Algorithms  can  be  studied  empirically  with  rcandoni  input. 
And  selection  cand  ranking  algorithms  can  serve  as  perfeart  hash  ftuictions. 

The  general-purpose  algorithms  of  this  chapter  c<apitalize  on  the  close  connection  be- 
twt'on  labelled  gr«aminars  and  generating  functions.  Since  the  algorithms  are  closely  related, 
we  will  begin  with  the  simple  counting  problem  and  then  explore  the  difficulties  presented 
by  more  complex  functions.  The  descriptions  below  care  restricted  to  labelled  formal  lan¬ 
guages  with  the  box  operator.  All  the  algorithms  generalize  to  more  complic«ated  partial 
orders,  but  the  details  wanild  obscure  the  bcisic  features. 

For  purposes  of  exposition  it  is  calso  helpful  to  restrict  the  grammars  so  tlnat  tlic  right 
sides  of  all  productions  have  cither  two  nonterminals  or  a  single  terminal.  The  start  non¬ 
terminal  is  the  only  nonterminal  that  can  derive  an  empty  string,  and  in  these  ciises,  the 
stfart  nonterminal  cannot  appear  on  the  right  side  of  any  production.  For  ordinary  formal 
languages,  a  grammar  in  this  restricted  format  is  considered  to  be  in  Chomsky  normal  form. 
There  arc  algorithms  to  remove  f-proditetions,  eliminate  chains  like  A  B,  D  Cj  and 
C  D,  and  break  up  larger  productions  that  will  expand  the  grammar  by  no  more  than 
squaring  the  number  of  productions,  and  will  not  introduce  ambiguity  into  an  tinambiguous 
grannutU*.  (See  [Harrison  1978;  Chapter  4|.)  These  same  algorithms  will  work  for  labelled 
grammars  as  long  /is  we  take  some  care  when  we  break  up  larger  productions  with  the  box 
operator.  The  production  P  QRS^TU,  for  examide,  is  equivalent  to 

P  QPf 
Pi  RP^ 

P2  S°Pz 

Pz^TU. 

In  general,  we  can  decompose  a  large  production  into  tany  binary  tree  of  two  nonterminal 
productions,  ami  then  box  every  nonterminal  on  the  path  thri)ngh  the  tree  to  tlic  boxed 
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dement  of  the  origiui'J  production.  If  boxes  cT-rc  handled  this  way,  then  the  traditional 
algorithms  for  converting  arbitrary  context  free  grammars  into  Chomsky  normal  forms  will 
also  normalize  labelled  grammars. 

4.1  Generation  Problems 


4.1.1  Counting 

Chapter  2  developed  the  direct  correspondence  between  labelled  grammars  and  integral 
equations  in  order  to  solve  the  equations  for  closed  form  generating  functions,  or  at  least 
to  recover  information  analytically  from  the  equations.  In  this  section  we  arc  interested  in 
solving  these  equations  mcchiuiically  for  sp(H:ific  terms  of  the  series  expansion.  This  can 
be  done  by  iterating  the  equations  (in  a  carefidly  chosen  order)  until  the  desired  solution 
is  reached,  but  before  wc  explore  the  method  of  iteration,  wc  need  to  understmid  the  basic 
computation  step.  Suppose  that  the  nonterminals  of  the  grammar  arc  and 

that  the  terminal  characters  x,  a,  and  b  are  critical  (x  being  the  special  labelled  character). 
Each  nonterminal  will  Inave  a  three  dimensional  series  expansion: 

P(z, a,b)  =  Y. 

},k,t 

with  integer  coefficients  in  a  matrix  P  associated  with  the  nonterminiil.  The  (j,  k,  1)  entry 
in  P  is  the  nnmber  of  different  strings  derivable  from  P  having  j  I’s,  k  a'a,  and  I  b's. 
A  production  like  P  -*  QR  indicates  that  the  convohition  of  the  Q  and  R  matrices  will 
contribute  to  the  terms  in  P: 

Pjkl  ♦-  Pjkl  +  ^  mnoPj  — n,/-o  *  (4.4) 

m,n,o  '  ' 

Notice  that  the  convolutioii  has  an  extra  factor  (^) .  IIcToaftcr  this  will  be  called  the  split 
factor,  since  it  accounts  for  the  splitting  of  labels  between  the  nonterminals.  In  general, 
the  split  factor  depends  on  the  box  operator.  If  C?  is  boxed  in  the  above  production  then 
the  factor  is  or  U  R  is  boxed,  the  f^tor  is  Boxing  removes  one  from  the 

corresponding  term  (m  or  j  -  m)  in  the  binomial  coeIRcient. 

Equation  (4.4)  above  highlights  one  of  the  difficulties  of  computation.  Pjkl  can  depend 
on  most  of  the  matrix  entries  in  Q  and  J?,  so  we  must  compute  all  the  matrices  together. 
One  way  to  avoid  conflict  is  to  compute  the  entries  in  an  order  that  always  increases  the 
sum  of  their  components.  Every  location  (j,  fc,l)  with  j  4*  fc  + 1  =  f  is  computed  before  any 
]o<*ation  with  j  +  =  f  +  However,  this  order  do(*s  not  resolve  all  the  problems  that 

can  arise.  Retuniing  again  to  I'quation  (4.4),  if  we  have  Qooo  nonzero,  then  Rjki  should  be 
computed  before  Pjki*  So  even  within  a  particular  location  we  must  follow  a  special,  safe, 
order  of  computation  among  the  nonterminals. 

To  find  the  safe  order  wc  identify  those  nonterminals,  like  Qooo  above,  that  can  derive 
strings  that  arc  free  of  critical  characters.  Such  strings  will  be  be  called  pseudo  empty. 
Algorithms  that  identify  nonterminals  that  derive  pseudo  empty  strings  are  well  known, 
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allhoiii’li  thry  appear  in  the  guise  of  eonijuiting  fiinrtional  dependenries  [Uliiiian  1980],  or 
tostinj^  for  oniptirn\ss  of  a  laiijjjuago  [Aho  J972]. 

Definition.  A  order  of  roinputation  is  a  Unear  embeddin^f^  of  tbv  following  partial 
order:  iV,  -<  Nj  (read  Ni  precedes  Nj)  if  there  is  a  production  Nj  NiN^  or 
Nj  — ►  and  derives  a  p.sTUffo  empty  string. 

If  dwve  is  a  cycle  in  the  partial  order,  e.g.  Ni  <  Nj  <  Nk  <  Ni,  then  tlie  count  is  ill  defined; 
the  cycle  generates  an  infinite  niunber  of  derivations  with  a  fixed  mnnber  of  occurrences  of 
the  critical  characters. 

Altogether,  we  have  described  the  following,  iterative,  ajjproach  to  computing  the 
nonterniinal  matrices: 

for  t  0  to  00  do 
for  j  ^  0  to  <  do 
for  fc  0  to  t  -  jf  do 
begin 

I  4^  t  —  j  -  k; 

for  each  nonterminal  P  according  to  the  safe  partial  order  do 
compute  Pjf^i  using  equation  (4.4); 
end 


The  running  time  of  this  algorithm  can  be  expressed  in  terms  of 

d  The  number  of  characters  critical  to  the  counting 
n  The  number  of  nonterminals 

m  The  maximum  number  of  occurrences  of  any  given  critical  character. 

For  most  applications  d  and  possibly  n  are  small  fixed  constants,  while  m  grows  large  and 
has  the  greatest  im])act  on  the  space  and  running  time  of  the  algorithm. 

The  parameter  d  is  the  dimensionality  of  the  problem.  Each  nonterminal  has  a  d- 
dimensional  array  of  coefficients,  with  tn  being  the  maximum  coordinate  in  «iny  dimension. 
Thus  the  space  re<iuired  is  0{nm^). 

The  time  necessary  to  fill  the  arrays  is  0{nm^^)  since  each  of  the  entries 

can  re<iuire  an  0(m‘')  convolution  of  two  series  whenever  there  is  a  production  with  two 
nonterminals  on  the  right  side. 
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4.1.2  Selection  and  Generation  at  Random 

The  next  step  in  diffirulty,  beyond  counting  derivations,  is  to  produce  strings  derived 
by  a  gratnin.ar.  If  '.vc  anticipate  that  the  typical  user  will  first  specify  a  family  of  objects 
with  a  graniiu.ar,  and  then  make  numerous  rc<iuests  for  random  or  iselected  members  of 
the  family,  then  our  efforts  should  lie  directed  at  stre.amlining  the  latter  process.  For  this 
re.a.son  the  exposition  of  the  section  is  reversed.  The  section  begins  with  the  d.^scriptioii  of 
a  n-.ilk  structure  that  is  well  adapted  to  random  generation  and  selection  problems.  Every 
combiimtori.ll  family  has  its  own  walk  structure  and  this  is  used  repeatedly  and  efficiently 
to  generate  strings.  Only  biter  in  the  section  will  it  become  clear  how  to  create  a  walk 
structure  from  a  grammar. 

Labelled  strings  are  generated  by  a  w.ilk  procedure  that  traverses  the  ilata  structure. 
The  procedure  has  two  arguments,  an  integer  c  specifying  the  string  (or  substring)  to  be 
generated,  and  a  list  of  labels  that  will  .appear  on  the  .special  characters  of  the  gener.ited 
string.  For  the  first  call  to  walk,  c  is  specified  by  the  user  (in  the  case  of  a  selection  request) 
or  gviierated  at  r.iiidom.  We  iiiiti,ilize  the  label  list  with  l...j  where  j  is  the  number  of 
spcH'ial  characters  expected  in  the  output. 

The  w.ilk  structure  is  a  bin.iry  tree  with  two  types  of  nodes:  trivial  .iiid  drastic.  All 
nodes  contain  an  integer  ti,  specifying  the  number  of  different  travers.ils  that  can  originate 
at  the  node,  and  possibly  two  pointers  to  ilescend.int  nodes.  Trivial  nodes  are  depicted  as 
di.imonds  in  the  figures  that  follow. 


Figure  4.1  TVivial  Nodes 

When  the  traversing  procedure  re.iches  a  trivial  node,  the  integer  carried  by  the  pro¬ 
cedure,  c,  shoidd  be  in  the  r.inge  0. . .  t;  -  I.  A  comparison  of  c  with  the  v.ilue  of  the  left 
descendant,  wj,  determines  the  direction  of  the  walk:  if  c  <  vj  the  procedure  traverses  the 
left  descend, int,  otherwise  c  is  replaced  by  c  -  vj  and  the  right  descendant  is  traversed. 

A  drastic  node  can  also  Imve  two  offspring,  biit  iti  this  c.ise  both  descendants  arc 
traversed.  Fmcli  drastic  node  correspomls  to  firing  a  production  in  the  origiiml  grammar, 
so  there  is  .i  protluction  right  hand  side  such  ;us  a  or  QR  .issoci.ited  with  the  node.  The 
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travoFi^ing  })rorc(lurt'  will  either  otitput  the  terminal  ehararter  a,  or  traverse  both  descen¬ 
dants  to  produce  stii)st rings  ft)r  Q  and  R,  in  which  case  there  .are  two  .additional  fields  // 
and  Ir  in  the  drastic  node;  tliey  .arc  tlie  iimnher  of  labels  given  to  Q  .and  R  respectively. 


Production  Used 


Figure  4.2  A  Dr.astic  Node 

Recall  that  when  j  spcci.al  characters  arc  expected  in  the  output,  the  top  level  call  to 
the  traversal  procedure  begins  with  a  link^  d  list  of  \  through  j.  At  a  drastic  nodes  with 
descciKhuits,  the  procedure  spots  the  current  list  of  labels  into  two  lists  for  tlic  desceiidcnts. 
If  a  box  operator  is  .applied  to  either  non^^'rmin.al  then  the  sni.allcst  label  is  moved  to  the 
designated  group  before  the  splitting  process. 

At  a  drastic  node  the  integer  c  carried  by  tlic  traversing  procedure  is  divided  into  three 
integers,  one  for  each  descendant,  and  one  to  control  the  splitting  of  the  labels: 

Cr  ^  C  mod  Vr 
C  C  div  Vr 

Cl  ^  c  mod  vi  '  ^ 

:i  c  div  vi . 

Using  that  portion  allocated  for  splitting,  s,  a  list  of  labels  can  be  partitioned  into  two 
groups  of  sizes  a,  and  b: 

while  a  -h  6  >  0  do 
if  5  <  then 

begin 

add  the  next  label  to  the  a  list; 

0  4-0-1; 
end 


begin 

« ^ -  (“aM*); 

add  the  next  label  to  the  b  list; 

b  < —  6—1; 

end 
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Till'  loop  pr(  >rrvrs  tiu'  invariant  that  a  -  lahrls  rvinaifi  to  hr  prorf^v.-r<l.  ami  i)  <  i*  < 

Sinri'  C*^^)  (*\,  *1^)  *  ("*  \i  *)  partitioT)  thr  raipv'  |().  parts,  ono 

rorn‘S)>on«liiu:  to  j)nttijp^  thr  lir>t  lahrl  into  thr  n  ;:rofip.  in  wliirli  rasr  thr  rc’inaining 
j)rol)]rni  is  to  partition  n  -  I  *  ^  hihrls  into  ’groups  of  si/r  a  -  1  and  h.  and  s  is  appropriately 
in  thr  lair^r  ^th  (*\,  j  har;trr  valn»  s  of  s  rorn'spond  to  pnttiip;  tin*  first  label  in  tlu-  h 

rronp.  and  so  s  will  hr  in  thr  ajipropriatr  ranrr  after  snhtraetin^ 
invariant  is  prt'srrvrd. 

Wr  now  have  all  the  in^n'dirnts  for  the  walk  prornlurc: 


procedure  u  alki  frrr,  r.  labrl.^) 
case  irrr^,  .uofle.  type  of 
trivuil: 

if  r  <  trcf* .If fC .value  then 
ra//  wnlk(  trrr*  .left.  t\  labch) 

else 

call  *.rf {//if,  c  “  tree* .left .value,  labcl.%): 

flr.istic: 
begin 

if  tin*  production  is  of  the  fonn  P  -*  a  then  oiifj>nf  a 
ehsc 
begin 

use  (  i.o)  fo  scpiir.itr  r  into  e^.  and  .s; 
if  a  ho.Y  appears  in  thr  prof  hut  ion  then 
n’move  f/ie  sn/riZ/esf  infe*f<T  from  labels: 
use  s  fo  .s/>/if  labels  into  left  labeis  mu!  nyht .hibeh: 
if  a  hi'.v  ap/nvirs  in  f/ie  prodnetion  then 

nTiirn  the  snudlot  /a/x'i  fo  thr  appropriate  gronp; 
ea//  U'(ilk{tTer]  .Ifjt.ci.  left  labels); 
ea/i  uvi/A(/rrt  t  .rt{//i/.  Cr.  right  Jahch): 
end 

So  till'  walk  procedure  is  a  strai;.;ht forward  l»yhrid  of  trt'e  search  and  tree  traversal. 
Trivial  nodes  are  “searchiMr  and  drastic  nodes  are  "traversc'd’*  in  order  to  g(*nerate  lal>elled 
terniinal  strings. 


Of  course  an  obvious  qnesiion  remains:  how  do  we  construct  a  w'alk  structure  for  a 
given  family  of  combinatorial  objects?  The  answer  is  surpri.dngly  simple;  we  use  the  algo¬ 
rithm  of  the  prect'iling  ,'iertion  for  counting  tlie  number  of  tihjerts  and  replace  the  additions 
and  multiplications  in  the  computation  with  sYrncfnr.d  additions  and  multiplications  that 
patch  togi  thiT  pieces  of  walk  structure. 


i  \  (;i:NrnATi(>N  ani)  Kr.('(){;NiTioN 

A  tiivial  noilr  results  from  the  a^]^]jtion  of  tw(»  nodes,  either  (Irastir  or  trivial.  The 
vahH'  iielti  oi  the  new  nodi'  :s  the  snin  (^f  tlie  value  fii'his  of  the  roinbined  nodes; 


Figure  4.3  Suniming  Two  Nodes 

A  drastic  node  is  the  result  of  the  multiplication  of  two  nodes: 


Figure  4.4  Multijdying  Two  Nodes 

Most  of  the  fields  of  the  drastic  node  (such  as  the  production  used  and  the  nuniher  of 
labels  going  h  ft  and  right  in  the  split)  are  provideil  by  the  C4)nt('xt  of  the  multiplication. 
The  sj)ht  factor  5  is  coniptited  according  to  the  inunher  of  labels  going  h  ft  and  right,  and 
the  valne  Hehi  of  tlie  ik'W  node  is  tin*  product  of  S  and  the  values  of  the  two  desceiid;uits. 

In  this  way,  llie  counting  procedure  of  the  preceding  .wtion  ran  be  ailapted  to  compile 
a  walk  structure.  To  improvt*  tin*  walk  struct un*.  it  is  advantagin)!is  to  postpone  summing 
the  nodi's  togiUher  as  di’scrilx'd  above.  Instead  of  immediately  creating  a  trivial  node 
whenever  a  sum  occurs  in  the  countitig  procedtirc\  we  first  rolleit  all  those  nodi*s  that  will 
eventually  he  part  of  the  same  sum.  Once  the  sum  is  comphde,  a  ba/anm/  walk  structure 
can  be  built  from  the  ro!lerte<l  nodes  for  guaranteed  logarithmic  soiarrh  time  on  a  single 
search,  or  Huffman  s  algorithm  (Knuth  1973;  Stxrtion  2.3.4. 5j  can  be  used  to  build  a  walk 
structure  that  is  optimal  wlnui  averagc<l  over  «ail  possible  s<*arche8. 
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Tilt'  rniiiiin^  tiiur  vinci  spacf  rt'tpiirt'iiK'iits  ft>r  the  st'h'et ion  mid  j^rnerrition  «'vt  r<\n- 
doni  jiius  fall  into  two  ri.isst's;  prr-rtnnpntation  to  rrcato  a  walk  stnirtnro  from  a 

j^raininar.  ainl  post -ronipntat ion  tt)  Kt'in  ratt'  ohjtrts  nsin^  the  walk  strnrtnre.  The  cost  of 
pre^ronipntatiim  is  based  on  the  counting:  al;;orithin  desenhet!  earlier.  The  time  rctpiired 
remains  and  hiran.se  earh  innlt  ipliration  and  athlition  ertvites  new  data  structure, 

tilt'  spact'  retpiirt'd  is  also  ()(nTn^^). 

Post -coinputat ion  requires  a  hit  more  reasonin'^:  since  each  <lrastic  node  corresponds 
to  the  tiring  of  a  pntdut  tion  in  the  j^rammar  anti  since  each  traversal  generates  a  string 
of  tht'  grammar,  the  traversal  will  visit  as  many  tlriistic  males  as  tliero  arc  productions 
conlnhutiiig  to  the  Huai  string.  If  the  final  string  is  of  length  L  then  we  can  use  at  most  I 
pnitluclions  generating  terminal  symbols,  and  at  most  I  prtuluctions  of  the  form  P  Q/?, 
since  eaeh  ntmterininal  must  generate  at  hvist  i>ne  terminal  So  the  string  generation  can 
recpiire  at  most  0(/)  productions,  and  the  traversal  can  visit  at  mf)st  0(1)  drastic  nodes, 

1'h(*  traversal  of  a  drastic  noth'  involves  a  splitting  operation  on  a  label  list  (0(m) 
time)  and  the  time  necessary  to  find  the  iit'Xt  ilrastic  iicnle.  Between  drastic  nodes  tlu're  is  a 
network  of  trivial  nodes  produced  by  the  summation  in  the  counting  algtirithm.  t>ince  there 
,ire  at  most  terms  summetl  together  (the  convolution  of  two  dalimensional  series) 

we  expert  to  visit  at  most  trivial  notles.  or  if  tlie  balancing  technique  described 

above  is  ummI,  O(dlnm)  trivial  nodes. 

Altogether  the  time  turessary  to  generate  a  string  is  0(l{m  +  dlnm)). 

4.1.3  Enumeration 

Our  next  an<l  last  problem  within  the  generation  category  i.s  the  systematic  onutnera- 
tion  of  all  iiK  inbers  of  a  combinatorial  family.  Ft)r  a  simple  solntion  to  this  problem  we  can 
use  the  M'hvtion  alg!>ritlim  of  the  pn'ce«liug  sertion,  sequencing  tin'  algorithm  through  all 
possible  values.  However,  since  most  iif  the  iliHereiu  es  betwirn  the  )th  and  (j  4  l)st  mem¬ 
bers  td  C4)nibinatorial  families  are  confiuc'd  to  small  regions  of  tlu'ir  .strings,  it  i.s  senseless 
to  com]»h‘lely  regenerate  the  strings.  The  nniK'ces.^ary  regeneration  is  avoided  by  loailing 
the  g<  neratetl  characters  into  a  hnlfer.  and  then  reusing  all  hut  the  right  end  of  the  buffer 
for  subsequent  strings. 

In  the  prect'iling  s<'ction  we  tlevelopnl  a  walk  algorithm  that  produced  a  single  string 
from  an  integer  input.  In  this  section  the  algorithm  that  emimerates  ail  strings  is  called 
ail- wvi/k.";  it  will  occiu^ionally  use  the  walk  algorithm  a  subroutine. 

Ali-wa/k.s  uses  the  data  structure  compileil  for  walking,  hut  it  treats  the  luxles  differ¬ 
ently.  At  a  trivial  node'  tlie  a//-walk.<  proce<lure  calls  it.<elf  riTUFMvely  on  both  ilescendants 
(recall  that  tin'  walk  pn>cedurr  choos<*s  only  one  <lescendant).  At  a  dnistic  node  the  pro- 
cedun'  uses  a  si't  of  lu'sled  loops  to  cover  all  possibilities: 

if  the  pnuhictum  is  of  the  form  P  a  tlicn  append  a  to  the  hii/fer,  and  ontpiit  the  buffer 
else  l]je  proflncfion  is  of  the  form  P  *  ♦  QR^  so  ... 
for  f’ill  /)o.s.sfM('  .spli/s  of  file  label  list  do 
begin 

for  n  0  to  vi  -  I  do 
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begin 

call  u'alk{frcr]Jfft,n.left  lnhel.s),  apprndirig  tliv  results  to  the  buffer; 
ealJ  oll-valks  o/i  f/u’  rig/;f  flcsceTuhint; 

end 

end 


4.2  Recognition  Problems 

\Vc  now  our  attention  from  the  generation  of  strings  to  the  recognition  of  strings 
supplied  as  inj)ut.  There  are  two  algorithms.  The  tiser  can  supply  a  string  and  a  grammar 
and  ask  the  system  to  verify  that  the  grammar  derives  the  string.  This  is  «accomplished 
with  the  accepting  algorithm.  There  is  also  a  ranking  algorithm  that  carries  this  process 
one  step  further;  it  returns  an  integer  that  tmicpiely  identities  the  string  within  the  family 
generated  !)y  the  grammar. 

4.2.1  Accepting 

Accepting  a  string  is  a  straight forw’anl  application  of  dynamic  programming,  a  tc<’h- 
nique  that  in; my  «iuthors  have  u.'^od  to  parse  context  free  languages.  We  assume  that  the 
iiiput  string  is  of  Icngtli  /.  and  we  construct  for  <'very  nojitermiiial  €111  (/  -f  1)  x  (/  -h  1) 
upper  triangular  matrix  with  entries  initially  hrlse.  The  entry  will  be  set  true  if  the 
nonttTininal  Ctau  derive  tlu'  piece  of  string  between  locations  i  and  7  “  1  inclusive. 

There  is  one  other  (/  f  1)  x  (/  -i-  1)  upper  triangiilar  matrix  A/,  called  the  minimums 
matrix,  whose  (L7)th  entry  contains  the  smallest  label  Ix^tweiMi  locations  i  and  j  -  1. 
Tlie  minimums  matrix  is  easily  computed  with  the  recurrence  A/(t,»  -=  min{A/(t,7  -  1), 

A/(t  +  l.j)h 

The  nonterminal  matrices  are  more  ditticidt  to  compute.  Their  entries  are  processed 
in  increasing  order  <»f  the  dilference  7  —  1.  Initially  the  near  diagonal  entries  {t,  i  4-  1)  arc  set 
true  for  any  nonterminal  with  a  production  P  a,  where  a  matches  the  input  at  location 
I.  The  nj)per  right  corners  are  computed  last.  A  true  entry  in  the  upper  right  corner  of 
the  start  rumtenninal  iiuatrix  indicates  a  successful  derivation. 

For  a  given  location  in  the  matrices.  (1,7),  all  matrices  arc  computed  together,  using 
the  safe  order  among  nontcuininals  describe<l  already  in  Si'ction  4.1.1.  Wlnui  a  n  mteiminal 
P  with  pnxliiction  P  QR  \s  processed,  the  production  is  matched  with  the  input  string 
hetwetm  t  <ancl  j  -  1,  w?t]i  the  understanding  that  Q  and  R  can  derive  variable  length 
.strings,  so  we  must  try  all  possible  midpoints  m  in  the  range  i  <  m  <  7  -  1.  An  outline  of 
the  algorithm  folKiws. 

for  ^  2  to  if. input)  do 

for  i  4—  1  to  lerigth(input)  -  3  -f  1  do 

begin 

7^1  +  5 

for  each  non  termini'll  P  in  safe  order  do 
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P{iJ) 

for  each  production  possibility  P  QR  do 

for  m  ♦-  i  +  1  to  j  -  1  do 

if  C?(i,  m)  and  R(m,j)  and  rijcrJc  box  operator  then 
begin 

P(iJ)  ^  true 
goto  found 
end 

found: 

end 

end 

The  prosi  nee  of  a  box  operator  adds  an  additional  check  to  the  procedure.  If  a  box  is 
on  a  nonterminal,  such  as  Q,  then  each  time  tlio  Q  matrix  entry  (t,  m)  is  tcste<l  for  true 
wc  also  check  if  Af(i,  m)  =  Equality  in.snres  that,  if  the  production  were  used  to 

derive  the  string  between  i  and  j  -  1,  then  the  snndlest  label  would  appear  in  the  proper 
location. 


The  running  time  of  the  accepting  algorithm  is  0{n(/-f  1)^),  where  I  is  the  length  of  the 
input  string,  and  n  is  the  number  of  nonterminals.  Each  of  the  0(n(l  +  J)^)  matrix  entries 
treated  by  the  algorithm  can  require  an  0(1)  string  match  in  the  inner  loop;  together  these 
derive  the  time  bound. 

4.2.2  Ranking 

The  ranking  algorithm  makes  use  of  the  following  data  structures.  The  first  two  are 
modifications  of  those  used  for  accepting: 

1)  Nonterminal  substring  matrices,  P(iJ)^  Previously  these  contained  true  or  false, 
dei)ending  on  whet’  er  tb.e  nonterminal  could  derive  the  string  betwmi  i  and  j  -  1.  Now 
they  contain  false  f)r  the  rank  of  the  derived  string.  We  .assume  that  the  grammar  is 
unambiguous;  ambiguous  grammars  would  require  a  s<'t  of  ranks. 

2)  Minimunw  matrix,  M(iJ).  This  used  to  contain  the  minimum  label  in  the  string 
between  i  .and  j  -  1,  Now  c.ach  entry  cout.ains  an  ordered  list  of  .all  the  labels  used  between 
t  .and  J  -  1,  with  the  smallest  label  first. 

The  third  data  structure  w.as  ti8e<l  before  in  counting: 

3)  Nontermimil  count  matrices,  fp.  These  cont.ain  the  tot;il  nninber  of  derivations 
that  st.art  with  the  nonterminal  P,  and  have  a  fin.al  string  with  rhar.acteristic  vector  p. 

4)  Production  off.'ct  matrices,  (P  QR)qr  This  is  a  new  data  structure  that  will 
be  explained  shortly.  Each  production  has  a  matrix  of  integers  th.at  is  addressed  by  two 
char.act eristic  vectors,  q  and  r,  corresponding  to  nontcrmin.als  on  the  riglit  side  of  the 
production. 
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Notr  tliat  tlu'rt'  arr  two  (li.  tinrtly  ilitfrrrnl  inatrirc's  assori.itiMl  with  oarh  nontt'rminal. 
TIu'  suhstriiij;  matrix  will  .iKvays  hr  two  chiiK  iisioiial  (siiur  it  is  a<ltlr(’ss(‘(]  l>y  two  huatioiis 
ill  thr  striii’').  aihl  will  hr  »lrnotril  with  thr  ir.dirrs  following;  tlir  noiitc'rniiiial  rharactrr: 

Tlir  comit  matrix  has  a  variahh'  numhrr  of  dimensions  d  drprndin;;  on  tlu’  nunihrr 
of  rliarartiTs  critiral  to  thr  ronntin*^.  It  is  denoted  with  snhsrriptrtl  indirrs  Pji^i  or  with 
a  vn  tor  as  a  subscript  /p.  The  subscript  in, distinjpiishrs  count  mat  rices  from  sul)string 
matrices. 

Th<’  arc('})ting  algorithm  of  Section  4.2.1  is  used  as  a  basis  for  thr  ranking  algorithm 
of  tliis  si'ction.  Instead  of  setting  P{irj)  true  in  the  inner  loop  of  the  code,  we  actually 
coni[)ute  the  rank  by  the  formula 

C(«.j)  -  Qq  (  g(.-.m))  /?r  +  f  {P  -  QRUr.  (4.G) 

where  most  of  the  above  formula  need;'  fm  tin  r  ('Xplanation.  In  essence,  the  formula  inverts 
the  ojirrations  of  Section  4.1.2  for  seleciinn  Wliereas  e»iuations  (4.5)  dividc’d  the  control 
intt'ger  c  into  thrtu'  parts  by  modular  ai  it  hm.  i  ic.  E(piation  (1.6)  above  assembdes  three 
parts,  s.  m).  and  n(rn.j)  into  one  integer. 

In  Sectitm  4.1.2,  .s  was  use<l  to  split  a  list  of  labels  into  two  groups.  Ih're  is  computed 
by  merging  two  groups  of  labels:  thosi'  ns(  d  in  the  Q  substring  (found  in  Af(z.  m))  and 
those  used  in  tlu*  /?  sul>string  (found  in  iV/(m,y)). 

s  4-  0; 

until  a  niul  lists  are  empty  do 

^^hcad  ^  ^liead 

begin 

mov'('  thr  <i  list  }u\'id  into  the  output 
a  a  -  1 
end 
else 
begin 

move  thr  h  list  head  mto  the  output 

I 

end 


This  accounts  for  <all  of  ecpiation  (4.6),  save  tlu*  last,  offset,  term  [P  Qflj^r  whose 
j)urpose  is  to  invert  tlu*  trivial  nodes  of  St'ction  4.1.2.  The  trc'es  of  trivia!  nod<*s  betw<vn 
drastic  nodes  nsult  from  a  summing  protess:  when  trivial  nodes  are  us<*d  fo:  seh’etion,  a 
search  based  on  the  control  integer  c:  guides  the  piogram  through  the  trivial  nodes  to  a 
particular  drastic  node,  and  the  tiring  of  a  sptviHc  production.  A  subrange  of  the  possible 
Willies  of  r  w  ill  all  land  at  tlu’  same  drastic  node.  In  the  inverse  ilirection.  \vc  have  ronij)uti*d 
an  integer  b»r  the  firing  of  a  prodnetum  and  we  must  achl  an  offset  for  the  base  of  the 
subrange.  The  offset  allows  for  the  contribution  of  other  terms  in  the  sum. 

Offset  matrices  can  be  computed  from  the  walk  structure  by  applying  the  procedure 
ronipute^oJJ'scUPpJ})  to  each  entry  of  the  nont«*rminal  count  matrices: 
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procedure  compute.ojf.^ct(trce,  3tim) 
case  tree] .node  Jype  do 
frivitii: 
begin 

call  compnte^ojfs€t(left  descendant^  sum); 
sum  ^  sum  +  value  v  at  current  node; 
call  compute. oJfset(7ight  descendantySum): 
end 
drastic: 

{product ioij)  of  ist  uontcrmjfrhar.  of  2nd  nontermj  ^ 


This  completes  the  inversion  of  the  selection  algorithm.  The  running  time  is  the  same 
as  tlie  accepting  algorithm,  0(n{l  4*  1)^),  since  the  only  significant  extra  computation  is 
the  merging  of  the  lists,  which  is  0(1)  and  so  can  be  absorbed  by  the  0(t)  time  used  to 
match  strings.  The  major  new  complication  of  the  ranking  algorithm  is  the  0(m^^)  sized 
offset  matrices;  it  is  ironic  that  the  simplest  nodes  of  Section  4.1.2  are  the  source  of  the 
most  complication  for  the  data  structures  of  this  section. 

4.3  Overall  Evaluation 

For  the  reader  interested  in  the  details  of  implementation,  there  is  a  PASCAL  version 
of  the  counting,  selection,  and  generation  at  random  algorithms  in  Appendix  C,  along  with 
a  sample  program  execution.  In  the  appendix  the  strict  requirement  for  Chomsky  normal 
form  is  relaxed,  to  allow  the  user  more  flexiblity  when  entering  grammars. 

Wliile  theoretically  the  running  times  and  space  requirements  of  the  algorithms  are 
all  polynomial,  the  implementation  experience  highlights  two  limitations  likely  to  cause 
trouble: 

1)  The  integer  size  of  the  host  computer.  Most  families  of  objects  grow  exponentially,  and 
so  even  the  simplest  counting  algorithm  must  use  large  integers.  Our  memory  estimates 
have  been  based  on  the  number  of  integers,  not  on  their  total  mnnber  of  bits. 

2)  The  memory  requirements.  This  becomes  particularly  acute  when  the  number  of  critical 
characters,  d,  exceeds  two,  so  that  the  memory  needed  for  selection  or  random  generation 
is  worse  than  O(m^),  where  m  is  the  maximum  number  of  occurrences  of  any  critical 
character. 

On  a  36  hi'  PD  P-10  computer,  with  256K  words  of  virtual  memory,  iuid  typicid  grammars 
from  Chap*  or  2,  these  limitations  where  provoked  when  m  12  «ind  d  «  3. 


The  algorithms  of  this  chapter  constitute  a  general  purpose  generation  and  recognition 
system  for  conibinatoriid  objects.  For  a  specific  problem,  si*ch  as  the  generation  of  random 
labelled  trees,  there  are  special  purpose  algorithms  that  will  do  better  than  the  general 
appro,ach.  If  ncccs.sary,  trees  can  be  genercated  in  0(m)  time  rather  than  O(m^)  time. 
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Novcrthrlr^^s  the  value  of  a  j;('iieral  systrni  is  clear:  rath<T  than  writ  in;;  i\  computer  program 
for  each  new  problem,  the  user  is  designing  a  ccunpaet  grammar.  In  fact  it  is  easy  to 
describe  coTubinatorial  families  with  grammars  of  fewer  tlian  8  productions  for  which  no 
special  purpose  algorithm  exists.  (See  the  sample  run  in  Appendix  C  for  trees  counted  by 
h'aves  and  single  descendant  nodes.) 

On  the  other  hand,  tlu’re  are  frcaiTU'works  for  selection,  enumeration,  and  generation  at 
random  that  are  more  general  tlian  the  lal)elled  grammars  described  above.  Most  notably, 
Wilf  has  dev<'l()p('d  the  idea  of  a  patli  through  a  directed  graph,  each  node  having  a  variable 
number  of  labelled  outward  cares  [Win  1977  and  1978]  Almost  cany  combiiicatorial  family 
with  a  recurrence  relation  can  be  place<l  in  one-to-one  correspondence  with  a  graph  that 
resembles  the  recurrence,  and  then  such  problems  as  generation  at  random  and  selection 
are  solved  with  genercal  purpose  algorithms  on  the  graph. 

The  chief  distinction  of  the  techiiicittes  of  this  chapter  are  that,  rather  tlnan  tising  a 
single  path  through  a  graph,  the  paths  are  allowed  to  fork  cat  certain,  drastic,  nodes.  While 
this  breanching  complicates  tlic  interna]  workings  of  the  algorithms,  it  permits  a  grccatly 
simplified  specification  of  the  conibiiicatorical  family,  and  essentially  automates  the  ouc-to- 
one  correspondence  hetw(vn  tnaversals  of  the  graph  and  combiiicatorial  objects. 


In  conclusion,  it  is  worth  cashing  a  more  reflective  question:  how  Ccan  the  polynomial 
algorithms  of  this  chapter  generate  objects  from  families  that  arc  inherently  exponential 
in  sizc.^  This  is  possible  btx'ause  of  the  high  degree  of  decoinposcability  present  in  families 
tluat  can  he  described  with  labelled  fornical  langUcages.  When  a  ii()nt<TminaI  N  appears 
in  ca  p*artia]ly  derived  .string  it  is  Irccated  in  a  manner  that  is  independent  of  the  size  and 
contents  of  the  string.  The  nature  of  the  enclosing  string  nmy  afr<*ct  the  babels  given  to 
the  string  derived  from  N,  but  the  processing  of  N  uses  a  arbitrary  list  of  labels  that  can 
contain  any  ordered  subset  of  the  integers.  This  independence,  ap<art  from  the  splitting 
of  labels,  makes  specical  trccatnient  for  substrings  unnecessary  - the  Wcalk  structure  can  he 
highly  folded,  with  numerous  pointers  to  snicall  subprohleins  that  arc  used  repeatedly  in 
the  construction  of  birger  objects.  This  is  the  source  of  the  Scavings  that  make  it  possible 
to  encode  an  exponentially  sizial  family  witli  a  polynoinical  sized  data  structure. 
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If  we  think  of  enumcrcition  problems  as  falling  into  three  broad  classes:  those  pertaining 
to  unlabollod  structures,  those  requiring  a  labelling  with  distinct  labels,  and  those  making 
repeated  use  of  labels,  then  the  intain  result  of  this  thesis  is  to  extend  the  use  of  formal 
languages  from  tmlabelled  to  distinctly  labelled  problems. 

Of  course  it  is  natural  to  ask  why  wc  bother  with  formal  languages  if  we  already  have 
combinatorial  techniques  to  deal  with  these  problems.  There  arc  several  reasons.  First,  the 
formal  language  approacli  is  general.  We  have  seen  already  in  Chapter  2  that  many  of  the 
chis.sic  generating  functions  can  be  derived  froni  labelled  formal  languages.  Second,  there 
is  an  extremely  close  connection  between  specification  cand  enumeration.  Once  wc  have  a 
forma]  language  description  wc  obtain  immediately  an  integral  equation  for  an  enumerating 
generating  function.  Formal  languages  have  bc'oii  extensively  used  for  specification,  often 
in  applications  where  enumeration  is  not  important,  so  it  is  natural  to  tie  a  large  class  of 
enumeration  problems  to  a  familiar  descriptive  tool. 

In  computer  science  tw^o  major  arefis  of  application  follow  from  the  connection  forged 
bctwcH}n  language  theory  and  enumeration.  We  can  use  labelled  formal  languages  as  analytic 
tools  in  the  antilysis  of  algorithms.  Chapter  3  developed  this  idea  on  two  sample  problems, 
the  second  of  which  is  a  new  algorithm  for  tree  .searching  that  is  interesting  in  it.s  own  right 
and  should  see  practical  use  in  appropriate  applications.  Another  major  use  for  labelled 
formal  languages  is  in  controlling  a  general  purpose  system  for  counting,  generation  at 
random,  and  enumeration  of  combinatorial  objects.  Chapter  4  describes  the  algorithms 
necessary  for  processing  labelled  formal  languages  and  appendix  C  includes  a  running 
system  of  this  nature.  It  is  no  longer  necessary  to  write  a  different  algorithm  for  each 
combinatorial  problem. 

Perliaps  the  largest  open  area  of  research  is  in  the  intermediate  problems  that  arc 
neither  milabolled  nor  distinctly  labelled,  but  make  repeated  use  of  a  collcH  tion  of  labels. 
Mathematically  these  problems  arc  uiiifi('d  by  the  Polya- Redfi eld  theory  of  enumeration 
(Polya  1937]  [Rcdfield  1927],  but  as  yet  there  is  no  systematic  way  to  generate  the  objects 
of  such  a  family;  Burnside’s  Icnnna  stands  between  the  counting  and  the  construction  of 
family  incinbcrs.  The  work  of  R.  C.  Read  is  a  significant  step  in  this  direction  [Read  1978]. 
His  approach  eliminates  many  of  the  redundant  family  members  before  they  are  checked 
for  independence.  But  there  arc  m<my  hist,  special  purpose,  algorithms  (sec  for  example 
aj>pendix  D)  that  challenge  us  to  pursue  this  problem  further. 


Tho  alj^nritliTijs  of  Cliaptor  4  scciri  ripe  for  asymptotic  iniprovcniont  of  thrir  running 
times,  lumg  and  TranI)  have  developt'd  me  thods  for  the  solution  of  polyneunial  <‘(|uation3 
that  imj)rove  the  running  time  of  the  counting  algorithm  of  Section  4.1.1  [Kung  19V8];  it 
would  !)('  nice  to  deveioj)  similar  techni(]ues  for  the  selection  and  generation  algorithms. 
The  accepting  problem  is  probably  equivalent  to  matrix  multiplication,  and  there  are  likely 
to  be  methods  that  work  C(Ui>iderably  fasten  on  limited  subsets  of  labelled  languages.  Both 
these  prohlems  need  further  exploration,  and  the  second  problem  suggests  yet  tanother  open 
area:  the  classification  of  labelled  grammars.  As  yet  there  is  no  easy  way  to  characterize 
those  languages  requiring  more  sophisticated  partidi  orders  than  those  denoted  with  the 
box  operator,  and  to  sej)aratc  the  various  uses  of  the  box  operator.  For  example,  the 
grammars  of  Sections  2.3.1  (alternating  permutations),  2.3.7  (Eulorian  numbers),  and  2.4.1 
(left  to  right  maxima  and  minima)  probably  all  belong  in  the  same  class,  and  yet  they  use 
ihe  box  operator  in  a  way  tliat  makes  them  clearly  beyond  the  classification  techniques  of 
ordinary  formal  languages. 


Appendix  A 


Bell  Polynomials  and  Lagrange  Inversion 


It  is  frequently  necessary  to  obtain  the  coefficients  of  a  generating  function  that  is  the 
composition  of  several  well  known  functions.  Suppose  that  h  =  f  o  g,  where  /,  g  and  h 
have  Taylor  expansions  like 

/=e4- 


«>i 


To  compute  h,-  in  terms  of  fi  and  gi  we  use  a  matrix  arrangement  of  the  Fa^  di  Bruno 
formula  introduced  by  Jabotinski  [Jabotinski  1947].  Each  function  /  has  a  matrix: 

mil  ”^12 


17122  ^23 

mss 


(A.2) 


where  rriij  is  an  exponential  Bell  polynomial  of  the  coefficients  of  /: 

mo  =  By<(/i,/j,...).  (A.3) 

(Note  the  unfortunate  transposition  of  indices.)  These  polynomials  are  defined  implicitly 
by  the  equation: 

gV/(.)  =  ^  By,(/i,  /s, .  . .)  »’^  *  (A-4) 

from  which  we  can  derive  various  other  expressions  for  Bjii 

Bii  =  f  (x*)  (/(•))* 

T  — i!— 

'■ 


(A.5) 


The  first  few  polynomials  are 


//i  h  h  •  -  X 

I  ^T:)- 


(AC) 


By  suniiuing  along  the  columns  we  obtain  another  family  of  polynoiniids  that  are  associated 
with  Bell’s  name: 

*>i 


The  top  row  of  the  matrix  contains  the  the  Taylor  expansion  of  /  and  each  successive 
row  can  be  computed  with  the  following  algorithm: 
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for  j  1  to  oo  do 
begin 
♦-  /; 

for  t  2  to  j  do 
begin 

TTlij  ^  0 

a  ^  j  (a  =  (^)  in  the  loop  below  ) 
for  fc  1  to  J  -  i  -{-  1  do 
begin 


o  ^  a 


idi 

k^i 


end 

mij  4-  TTiij/i 

end 

end 

The  most  useful  property  of  these  matrices  is  given  by  the  following  theorem. 

Theorem.  If  Mj  aiul  Mg  arc  the  matrices  of  f  and  g  then  =  M/Mg  is  the  matrix 
for  the  composition,  h  =  fog. 

Proof: 

r.- 


id 

id 

id.k 


(A.8) 


i,k 


For  the  Taylor  expansion  of  h  only  the  top  row  of  need  be  computed: 
Corollary.  Ifh  =  fog  then 


(A.9) 


Theorem.  If  n  function  f  is  given  implicitly  by  h[f)  =  z  then  the  mntrx  for  f  is 
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Proof:  [Comtct  1974;  p.  149j  Conaklor  the  product  of  Mh  •aJid  Mj,  where 


hy  equation  (A. 5), 


=  r  ~  !\t' 


ml 


(A.11) 


(A.12) 


Corollary.  (Lagrange  inversion)  If  a  function  f  is  given  implicitly  by  h(f)  =  ^  then 
the  coefScients  in  the  power  scries  expansion  for  f  are: 


(A.13) 


Here  are  two  frequently  encountered  matrices: 

/(x)=e»-l 

(  {}}  {?} 

Mf  =1  1^1  Stirling  Numbers,  2nd  Kind 


<;(x)  = 


ln(l  -  x) 

[{]  [?] 
Kl 


Stirling  Numbers,  1st  Kind 


(A.14) 


As  an  example  of  the  teclmjques  of  this  appendix,  consid jr  the  following  equations 
from  Section  2.3.4:  _ 

=  'A  151 


Appendix  B 


Diminished  Tree  Search 


1.  Arbitrary  Median  Tree  Searching.  The  following  algorithm  combines  a  tree 
search  with  a  linear  search.  When  the  tree  search  portion  of  the  code  hits  a  leaf,  it  branches 
to  additional  code  to  search  a  small  linear  list  of  size  at  most  2t.  If  insertions  cause  the 
linear  list  to  exceed  size  2t  then  a  new  tree  node  is  created  and  the  list  is  split  into  two  lists 
of  size  t.  The  additional  tree  balancing  provided  by  the  median  of  2t  +  1  splitting  improves 
the  miming  time  of  the  algorithm. 

define  plusjnf  =  mnxint 

define  half.sizr.  =3  {  This  is  t.  } 

define  median  =  half. size  -h  1 

define  overflow  =  half. size  -h  half. size  -h  2 

program  seafch[tty^  o\tpj  ); 

type  link  =  ]node\  node. type  (>cc,/i3f); 

node  =  record  case  node.iype  of 

tree:  (left  :  link]  key  :  integer;  right  :  link); 

list:  (keys  :  array  [1  ..  overflow  -  1]  of  integer); 

end; 

var  p:  link;  in.key:  integer; 

(Declare  the  Initialize  Procedure  e); 

(Docla»*e  the  Find  or  Lisert  Procedure  2); 
begin  initialize  (p); 
while  true  do 

begin  'Key>u');  read ( ffy ,  m.Aicy);  writeln{tty,find(in.key,p)); 

end; 
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2.  A  Sfandcird  Troc  Si'ar'‘li  Algorithm. 

Tlu'  find  proct'diuT  a  koy.  A:,  and  a  pointer  to  a  tree,  p.  If  k  is  in  the  tree,  then 

find  returns  true,  otherwisi* //dsc  is  returned,  and  k  is  inserted  into  tlie  tree.  Most  of  the 
work  is  performed  hy  the  inner  loop  of  a  standard  tree  search  algorithm.  8ince  the  loop 
is  included  without  modification,  any  improvements  that  the  programmer  might  design  for 
standard  tree  soiarching  (su<*h  as  hand  coding  the  inner  looj))  will  also  be  improvements  to 
diminished  tree  searching. 

In  order  to  activate  the  linear  list  co<le  without  modifying  this  loop  the  leaves  of  the 
tree  contain  a  dummy  -foe  key.  a  nil  left  pointer,  and  a  right  pointer  to  a  linear  list.  The 
standard  tree  seaich  will  find  the  nil  pointer  and  take  the  ^unsucccssfur  loop  exit, 
define  success  =  1 
define  quit  =  2 

(Declare  the  Find  or  Lisert  Procedure  2)  = 
function  Jind{k  :  integer:  p  :  link):  boolean; 
label  success ,  quit; 

vari.icmp:  integer:  Ic/t.listJeftJree,  rightJist^righUree:  link; 
begin  while  true  do 
with  PI  do 

begin  if  k  >  keg  then 
begin  p  right; 
end 

else  begin  if  k  =  key  then  goto  success 

else  if  left  =  nil  then  (Search  the  right  linear  list,  and  goto  success  or  quit  3) 
else  p*- left; 
end; 
end; 

success:  find  true; 
quit:  end 

This  coflo  is  in  section  1, 
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3.  Searching  a  Linear  List. 

The  following  code  is  a  standard  linear  list  search.  It  is  designed  to  work  for  arbitrary  list 
sizes  fcontrollcd  by  the  half. size  parameter).  Even  though  this  code  is  executed  only  once 
for  each  search,  as  opposed  to  O(Iogn)  passes  through  the  loop  in  the  preceding  section, 
the  prograrmner  should  take  care  to  make  this  run  efficiently;  logn  is  close  to  a  constant. 
In  particular,  the  code  below  is  NOT  THE  BEST  approach  to  the  problem  since  half.size 
is  fixed  during  execution.  It  is  better  to  choose  a  fixed  half.size  and  unroll  the  loop,  as 
exhibited  in  the  special  median  of  five  variation  following  this  code. 

(Search  the  right  linear  list,  and  goto  success  or  quit  3)  = 
begin  t  ♦-  1; 
with  right]  do 

begin  while  true  do 

begin  if  keys[i]  <  k  then 
begin  i  t  +  1; 
end 

else  begin  if  =  k  then  goto  success 

else  begin  (Insert  k  and  Shift  Subsequent  Keys  4)’ 
find  *—  false;  goto  quit; 
end; 
end; 
end; 
end; 
end 

This  code  is  used  in  section  2. 

4.  Inserting  a  Key  into  a  Linear  List. 

( Insert  k  and  Shift  Sub.sequent  Keys  4 )  = 
begin  repeat  temp  ^  k;  k  A:cy3[t];  keys[i]  temp;  t  4—  i  +  1 
until  k  =  plusJnf ; 

if  I  =  overflow  then  ( Split  the  Linear  List  5 ) 

else  A:cy.i[i]  ♦—  plusJnf; 

end 

This  code  is  used  in  section  3. 
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5.  Splitting  a  Linear  List. 

A  l.neai  list  has  grown  to  size  2  ♦  half. size  4- 1  so  it  is  split.  The  median  element  beronios 
a  new  tree  node  with  right  and  left  dummy  nodes  for  sons.  The  dummy  nodes  each  point, 
to  lists  of  half. size  keys. 

(Split  the  Linear  List  s)  = 
with  p]  do 

begin  IcftJist  ^  right',  key  ^  left. list], key s\median\\ 
left.list] ,key3\median\  4-  plus.inf;  new  (right.list,  list); 
for  t  <—  1  to  half  size  do 

begin  right. list], ke7j3[i]  4~  leftJist] ,k€ys[median  +  i]; 
end; 

right.list],key3[median]  ^  plus. inf ;  new  {left ,  tree);  new  {right  Jree); 
leftlkey  *-plus.mf  -,  righttkey  ^  plu.i.inf  -,  leftlleft  -  nil;  righttUft  —  nil; 
left], right  *- left. list-,  right], right  righUist] 
end 

This  code  is  used  in  section  4. 

6.  Initialize  the  d<ita  structure  with  a  tree  node  and  an  empty  list. 

(Decl  .re  the  Initialize  Procedure  6)  = 
pror  jdure  initialize  {\ at p  :  link); 

b  jgin  new{p,tree);  p],key  ^  plus.tnf;  p].left  ^  nil;  new {p] .right ,  list); 

4--  plus.inf  ; 

end 

This  code  is  used  in  section  1. 


MEDIAN-OF-FIVE  TREE  SEARCH 


1.  Median-of-Fivc  Tree  Search, 
define  plusJnf  ~  rnaxint 
define  reps  =  5 

program  seorch{tty^  ouipu^); 

type  link  =  ]node;  node.type  =  (tree^list);  status  =  ..  'e';  node  = 

record  case  nodeJype  of 
tree:  [left  :  link;  key  :  integer;  right  :  link); 
list:  (nexi.free  :  status;  a^h^c,d  :  integer); 
end; 

var  t:  link;  in.key:  integer; 

heap  ^bottom  y  time ,  tesLsize ,  start ,  test^number ,  j ,  tesLexponent :  integer ; 
result:  boolean;  ( Initialize  Procedure  2); 

(Find  or  Insert  Procedure  3}; 
begin  mnrk  (heap. bottom); 

writeln (output y  'SpecialuMedianuOf uFive rcp5  :  6,  'uu^epsyof ucachuSlze '); 
test. size  ^  1; 

for  test.exponent  ^  1  to  14  do 

begin  test. size  ^  2  ♦  test. size;  start  ♦-  runhmc; 
for  test.number  1  to  reps  do 
begin  msha/tzc(0; 

for  y  ♦“  1  to  test.size  do  result  ^nd(<rttnc (random (0)  ♦  (p/ti3_fn/  -  1)),05 

release  (heap. bottom); 

end; 

time  ^  runtime;  torifc/n  (output,  'TestuSize:  *y  test.size  :  5,  'uTime: 

(time  -  start) :  7,  "uTime/Size :  %  ((time  -  start) f  test.size)  :  8  :  4); 

end; 

end. 

2.  Initialize  the  data  structure  with  a  tree  node  and  an  empty  list. 

(Initialize  Procedure  2)  = 
procedure  initia/uc (var t  :  /ini;); 
begin  new (t,  tree); 
with  tt  do 

begin  key  plus.inf;  left  ^  nil;  neu;(riifAt,/i5t);  right], next. free  "'a'; 

end; 
end 

This  code  is  used  in  section  1. 
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3.  A  St«'indarcl  Tree*  Search  Algorithm, 
define  .success  =  1 
define  quit  —  2 

define  a.a  ~  101  {  These  labels  will  be  used  in  the  block  move  code} 

define  a.b  —  102 

define  a_c  =  103 

define  a.d  =  104 

define  a.c  -  105 

define  6.6  —  106 

define  6.c  107 

define  6.d  =  108 

define  b^e  =  109 

define  c.c  =  110 

define  c.d  =  111 

define  r.c  =  112 

define  d.d  113 

define  =  114 

define  c.c  =  115 

define  wrap.up  =  120 

(Find  or  Insert  Procedure  3)  = 
function  find{k  :  integer]  t  :  link):  boolean; 
label  success,  a_a,  a_6,  a.c,  a.d,  o.c,  6,6,  6«c,  6-d^  6. c,  c_c,  c.d,  dd  dc  ee 
wrap-up; 

var  right. list:  link;  e:  integer; 
begin  while  true  do 
with  /t  do 

begin  if  >  key  then  t  ^  right 
else  begin  if  fc  =  key  then  goto  5ticcc33 

else  if  left  =  nil  then  ( Linear  List  Search  4 ) 
else  begin  t  ^  left; 
end; 

end; 

end; 

^access:  find  ^  true; 
quit:  end 

Tliis  code  is  used  in  section  1. 


MEDIAN-OF-FIVE  TREE  SEARCH  03 

4.  Searching  the  Accumulator.  Since  the  search  loop  is  unrolled,  a  binary  search  is  easy 
to  implement: 

( Linear  List  Search  4 )  = 
with  right]  do 

begin  case  next.free  of 
'a':  goto  a.a; 

'b':  begin  if  A:  <  o  then  goto  a.5; 
if  A:  =  a  then  goto  success 
else  goto  6.6 
end; 

'c':  if  A:  <  6  then 

begin  if  A:  <  a  then  goto  a.c; 
if  Ac  =  a  then  goto  success 
else  goto  6.C 
end 

else  if  A:  =  6  then  goto  success 
else  goto  c-c; 

'd':  if  k  <  b  then 

begin  if  A:  <  a  then  goto  a.d; 
if  Ac  =  a  then  goto  success 
else  goto  6_(f 
end 

else  begin  if  A:  <  c  then 

begin  if  Ac  =  6  then  goto  success 

else  goto  c.d 

end 

else  begin  if  A;  =  c  then  goto  success 
else  goto  d.d 
end 
end; 

if  Ac  <  c  then 
begin  if  Ac  <  6  then 
begin  if  Ac  <  a  then  goto  o.e; 
if  Ac  =  a  then  goto  success 
else  goto  b.e 
end 

else  if  Ac  =  6  then  goto  success 
else  goto  c.e; 
end 

else  begin  if  Ac  <  d  then 

begin  if  Ac  =  c  then  goto  success 

else  goto  fLe 

end 

else  begin  if  Ac  =  d  then  goto  success 
else  goto  e.e 
end 
end; 

end; 


Appendix  C 

A  General-Purpose  Generator  of  Combinatorial  Objects 


!•  Global  Procedures  and  Parameters. 

This  program  accepts  a  graininatical  description  of  a  family  of  combinatorial  objects, 
counts  the  number  of  specified  objects,  and  generates  objects  at  random  or  by  rank  within 
the  family.  The  grammar  must  be  context  free,  but  it  can  contain  additional  “box” 
operators  to  control  the  labelling  of  one  of  the  terminal  characters,  so  that  .such  things 
as  labelled  trees  and  permutations  can  be  described  with  grammars.  The  reader  should  be 
familiar  with  the  “box”  operator. 

There  are  three  major  divisions  in  the  code.  One  section  loads  a  grammar  into  the  pro¬ 
gram’s  data  stnictures.  Another  section  counts  the  number  of  derivations  of  the  grammar 
by  a  process  that  amounts  to  tensor  multiplication,  and  at  the  same  time  generates  a  “walk” 
stnicturc.  The  third  section  traverses  this  walk  structure  and  generates  a  specified  string. 
The  data  structures  and  low  level  support  procedures  are  shared  by  all  three  sections  of 
code. 

By  convention,  the  single  letter  prefix  of  an  identifier,  e.g.,  x.variable.name,  groups 
together  common  variables  and  types.  The  following  interpretations  should  help  the  reader: 

i  input 

s  symbol  table 

p  production  possibility 

m  nonterminal  matrices 

w  walk  structure 

I  label  list 

program  general  (tty); 
type  (Global  types  8) 
var  ( Global  variables  3 ) 

{ Global  procedures  4 ) 

( Matrix  procedures  36 ) 

(Walk  structure  builders  45) 

(Griunmar  loading  procedures  11 ) 

( Counting  procedures  49 ) 

( Walking  procedures  58 ) 

(Command  processing  63} 
begin  initialize;  commands; 
end. 
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2.  \ari<)ii.^  parainrttTs  that  control  t!ic  rapacity  of  tlic  program  arc  defined  here, 
define  bitjjrr  ..stzf  -  MO  |  Maxiiniiiii  iiiirnbcr  of  char  on  an  input  line} 
define  rnax^prod  .symbols  7  {Symbols  in  a  production  possibility  } 

define  rrmi.cnuntnblcJermirials  ~  3  {  Terminals  critical  in  counting  } 

define  rnaiJotnl  -  12  {  Total  occurrences  of  each  critical  cliar  in  the  final  strin 

3.  The  mult  array  will  contain  the  multinomial  coefficients,  defined  by 


multli.j, 


(t-hj-f  fc)! 
i!  j\  Jt! 


(Global  variables  3)  = 

mult:  array  [-1  . .  1,  -1  . .  maxJotal,  -  1  . .  max.total]  of  integer; 
Sco  also  sections  3,  9.  22,  and  43. 

This  co<le  is  used  in  section  1. 


4.  Initialization  of  the  multinomial  array.  The  recurrence 


V»  j  k)  j 


is  used  to  create  [t,  j,  Jt]. 

( Global  procedures  4 )  = 
procedure  initialize; 
vari, J, f:  integer; 
begin  for  t  * - 1  to  maxJotal  do 

begin  mu// mu// [0, -l,i]  ^  0;  mu//[l,t, -l)  ^  0;  mu//[l,-l,i] 

for  J  < - 1  to  max.total  do  mu//[— *—  0; 

end: 

0. 0|  4--  1; 

for  t  ^  I  to  maxJotal  do 
for  t  ^  0  to  1  do 
for  y  ^  0  to  f  -  »  do 

mult  [«• , 7,  / - ,■  -j]  mult [j  - 1 ,  j .  / - 1 - j]  +  mult  [»',  j - 1,  / - - j J + rnult  [/,  j,t  - 

end; 

Sec  also  sections  6,  7,  23,  24,  and  57. 

This  code  is  usctl  in  section  1. 


5*  BulFer  liiid  variables  for  input. 

(Global  variables  3) 

i.hujjer:  packed  array  [l  . .  buffer.xize]  of  char-, 

i.scan.i.line.Mze:  integer;  {  Current  mkI  final  positions  in  the  buffer} 

help. file:  file  of  char;  {  A  place  to  find  help  } 
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6.  Here  arc  two  fimdainnital  opcratioria.  The  macro  eapitaLktter  is  true  when  its 
argmnont  is  a  capital  char  and  the  function  mtn  returns  its  smallest  parameter. 

define  rapfta/Jirtfer (#)  H  ((#  >  'A')  A  {#  <  '2')) 

(Global  procedures  4)  +  = 
function  :  integer):  integer; 

begin  if  i  <  j  then  min  i 
else  mm  ^  j; 
end; 

7.  The  procedure  i_h*nc  rciads  a  line  of  input  into  the  global  buffer,  discarding  blanks  and 
commas.  If  letters^only  is  true,  the  remaining  characters  must  be  capital  letters.  The  last 
character  is  stored  at  loc€ation  i .line, size  in  i. buffer. 

define  start.over  =  3 

{ Global  procedures  4 )  += 
procedure  i .line {letters. only  :  boolean); 
label  start.over; 

var  nciy.scan,  oriyinG/.scan:  integer; 

begin  start.over:  if  €oln{tty)  then  readln{tty);  {A  quirk  of  tty  input} 
read{tty,i. buffer  :  iJine.size);  oriyino/.scan  1;  new.scan  ^  1; 
while  original.scan  <  i. line. size  do 
begin  i.biiffer[new.scan]  ^  i. buffer  [or iginaLscan]; 
if  ->(i.buffer  [original.scan]  G  ['u%  '])  then 

if  Ictters.only  A  -{capital.lett€r(i.buff€r[new.scan]))  then 
begin  xvrite[tty,  'Error :ucapitaluletter(8)uexpected,utryuagainu. .  -u"); 
goto  start.over; 
end 

else  nexv.scan  new.scan  +  1; 
original.scan  ^  original.scan  +  1; 
end; 

iJine.size  new.scan  -  1;  i.scan  ^  1; 
end; 
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8.  Cjlub.il  D;jta  Sti ucturinj^. 

The  tl.it.i  stnirtnrr  is  a  synibol  UMr  tliaf  is  itirlrxrd  hy  rai>ital  letters,  aiul 

coiifaiM'^  iiiforniat ion  al'uut  the  letti'rs  Ti>e(l  in  the'  j^ranunar.  An  entry  for  a  nonter?niiiaI 
n  the  symbol  t.iMe  lias  sev(Ta]  point(Ts.  Tlie  first  {jirodr]  jioints  tf>  a  list  /)f  profjnrt ion 
possibilities.  If,  for  example.  A'  ran  be  n'wriften  in  thn’o  ways,  5,  T,  and  f/, 

iv  ->  .V  j  r  I  u, 

then  tluTe  will  be  a  linked  list  of  three  p  r^'yht.side  records  hanj^in"  from  the  profit  field  of 
the  A  entry  in  the  symbol  talile.  The  secfmd  (mafrijr)  points  to  a  ninltidimensirj,Hl  array 
that  is  ind(‘xed  by  the  ocmrrences  of  characters  critical  to  the  counting.  Suppose  the  user 
has  declan'd  A'  to  he  a  special  lalu-llecl  character,  and  F  and  G  to  he  countable  terminal 
symbols  W(  can  find  out  the  numluT  of  strings  derived  from  N  that  have  G  A'*s,  5  F’s 
and  d  (t  s  by  looking  at  the  (6.5,1)  entry  in  the  matrix  associatefl  with  N  in  the  symbol 
table.  If  we  wiTe  simply  intc'rc'sted  in  con  itiiig  these  occiirrc'nces,  then  an  integer  entry 
in  this  matrix  would  be  adi'que.te.  llowi'Ver,  vve  also  want  to  generatt*  object .s  with  the 
spcciti('d  numbers  of  terminal  symbols.  This  is  accomplished  with  a  walk  structure  that 
will  he  descrilxMi  later.  Each  matrix  entry  is  a  pointer  to  the  walk  structure  (w.ptr).  The 
integer  count  of  derived  objects  can  be  found  in  the  value  field  of  the  fir.st  walk  structure 
node  («;_n«de). 

The  additional  fields  of  a  nonterminal  in  the  symbol  table  (appears.in,  derives. empty , 
preceded. by,  and  followers)  arc  used  to  derive  the  safe  order  of  computation.  This  is 
an  oriler  among  nouti'rminals  that  will  he  (iescrihed  in  detail  later.  The  appears. in  field 
contains  a  linked  list  tliat  iioints  to  all  j)roductioTis  containing  the  nonterminal;  it  is  a 
reverse  directory  for  the  nonterminals.  The  boolean  derives.ernpty  will  be  sot  true  once  it 
is  determined  that  the  nonterminal  can  derive  a  pseudo  empty  string  (free  of  all  countable 
characters).  The  last  step  in  the  computation  of  the  .‘^afe  order  is  a  topological  sort  of  a 
partial  ordeT  among  the  nonterminals.  Tliis  partial  order  is  repn'sent.’d  by  followers^  a 
linked  list  of  all  nonterininals  that  must  follow  in  the  partial  order,  and  preceden.hy,  an 
integer  counting  the  occurrences  of  the  nontcrinina!  in  other  followers  lists. 

(Global  typos  8)  = 

a.ptr  =r  ^a.list:  c.ptr  ^  ]c.list\  m.ptr  =  fm.oxis:  p.ptr  =  ]p. right. side; 

w.ptr  =  ^w.node; 

sJype  =  (undefined,  uncouut.term,  count  .term  Jabelled ,  nonterm)] 

s.data  =  record  case  status  :  s  type  of 
labelled,  count.tcrm:  (index  :  infe^er); 

nonferm:  (prods  :  p.ptr:  matrix  :  m.ptr;  appears.in  :  a.ptr:  derives.ernpty  :  boolean; 
preceded. by  ;  integer;  followers  :  c.ptr); 

end; 

Sop  ;iJs<>  sc'tiioiis  10,  21,  35.  39,  tuhI  50. 

This  colo  is  nsoil  in  .section  1. 
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9.  llorc  is  the  actual  symbol  table  (sJable)  and  an  array  containiiif'  tliosc  terminal 
symbols  critical  to  the  countinj^  [index. symbols).  The  first  element  of  index. symbols  is 
€always  the  special  labelled  character,  and  nurnber. indices  records  the  total  number  of  entries 
in  index. symbols .  For  the  example  of  the  preceding  section,  index.symbols  would  contain 
X,  F  and  G  and  number. indices  would  be  three.  The  s .table  is  initialized  before  each  new 
grammar  is  input. 

(Global  variables  3)-f  — 

index  symbols:  array  [1  max.countable .terminals]  of  char; 

number  .indices :  integer; 

s. table:  array  ["A'  ..  'Z']  of  s.data; 

10.  The  production  possibility  data  structure. 

Suppose  that  A"  is  the  labelled  character,  F  and  G  are  counted  tcrininals,  //  is  an 
tincounted  terminal,  and  A  and  D  are  nonterminals,  Th.n  the  production 

N  XFA^GDGH 


generates  the  following  record: 

1)  string  contains  the  characters  XFAGDGH  and  size  is  set  to  the  length  of  string^  in 
this  case  7. 

2)  sub. string  includes  only  the  nonterminals  AD.,  and  sub.size  is  set  to  2. 

3)  bid  is  the  location  of  the  box  operator  in  string.  the  above  production  the  third 
character  is  boxed. 

4)  specinLbxd,  JirstJud,  and  secotuLbxd  are  all  either  zero  or  one,  depending  on  whether 
the  sptrial  character,  the  Irst  nonterminal,  or  the  second  nonterminal  tare  boxed.  Here  we 
have  Jirst.bxd  e<jual  to  one.  and  the  others  zero. 

5)  adjn.st  contains  1,  1,  and  2,  corresponding  to  1  X,  1  F,  and  2  G's  in  the  string.  The 
adjust  array  summarizes  the  effect  of  the  production  on  the  counts  of  index.sywbols  in  the 
dcTived  string. 

C)  p. left. side  is  N. 

7)  contributors  is  a  coimt  of  tlie  number  of  characters  in  the  string  that  arc  counted 
terminals,  jdus  the  number  of  nonterminals  that  might  contribute  countable  characters. 
In  this  exami)le  nmtributors  is  6. 

((Global  types  s)  = 

characteristic. vector  array  [1  .,  max.countable .terminals]  of  integer; 
p.right.side  =  record  size:  integer; 

.string:  array  [I  ..  max. prod. symbols]  of  char; 
bxd ,  specinL bxd ,  Jir.st.bxd ,  second.bxd :  integer ; 
adjust:  characteristic. vector; 

S7ib.size :  integer ; 

suh.siring:  array  (1  •«  2]  of  char; 

p. left. side:  char; 

contributors:  integer; 

p.next:  y.ptr 

end; 
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11.  Loading  a  Grammar. 

define  done  .loading  .gramvmr  =  1 
define  flush. produrtion  —  2 
define  '^.nnejjcitinj. productions  =  3 

{Grammar  loading  procedures  ll)  = 
procedure  loading  .grammar ; 

label  done  .loading  .grammar  ^  flush. production^  done. getting  ^productions] 
yar  left. Side .  cur. char :  char]  p.pnssibility:  p.ptr;  i:  integer] 

(Local  variables  for  computing  the  safe  order  25) 

(Local  variables  for  loading  matrix  entries  54} 

(Internal  grammar  loading  procedures  20 ) 

( Safe  order  procedure  33 ) 
begin  (Initialize  for  computing  the  safe  order  28); 

(Get  declarations  12); 
ivritcln(tty);  (Get  productions  14); 
writcln{tUj);  (Compute  the  safe  order  29); 
do7i€.l()ading.grnmmar:  end; 

This  code  is  uschI  in  section  1. 
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12.  Tlio  declaration  portion  of  loading  a  grammar. 

TIic  U'^er  is  asked  to  classif\  ♦’lo  letters  of  the  grammar  into  four  rategories.  A  letter  is  a 
nonterminal  c.r  a  terminal  t'  ss  labelled,  counted,  or  uncounted.  Note  that  the  user  must 
supply  a  sii.,:'  labelled  chaias’ter,  even  for  unlabelled  grammars,  and  that  E  is  a  built-in 
empty  string.  (Tlie  user  dc'  ^  v  need  to  say  E,  but  E\s  will  not  be  output  in  a  generated 
strins.) 

{ Get  (leclarations  12 )  = 

uncouniAcTm\  , 'LabelleduCharacter>>u'); 

iJtne(tTue); 

wh  dc  iJine^size  /  1  do 
f»Ggin  v)rite[tty^ 

'EnteruauSingleulabelleduCharacteru(duinmyuifunece68ary)u.  »  -u'); 
iJine  (trtxc); 
end; 

index  .symbol s\l]  t.6u^er[l]; 

with  ,s_to6/e[i_6uJ/‘cr(l)j  do 

begin  (Reject  double  dotinitions  by  going  to  doneJoading^grammar  13); 
status  labelled;  index  1; 

end; 

tvTite(tty^  'Coun+eduTenninal(8)»u');  i./inc(triie); 
if  iJine.size  >  max^countablejerminals  then 

writeln[tty,  'Error tutheul as tutwoulinesuhaveumoreuthanu', 
mar  countable Jerminaii  :  0,  'ucharactersucriticalutoutheucounting'); 
goto  done Joading. grammar; 
end; 

while  i.scan  <  iJine.size  do 

begin  index. symbols[i.scan  -f  1]  ^  i, buff er[i ..scan]; 
with  sJable[i.bnffer  [i.scan]]  do 

begin  (Reject  double  <lefinitions  by  going  to  done  .loading. grammar  13); 
status  count.term;  index  ^  i.scan  -h  1; 

end; 

i.scrtn  i_5can  -h  1; 
end; 

number. indices  ^  i.scan; 

write  {tty,  'UncounteduTenninal(8)»u');  i.line(true); 
while  i.scan  <  i.line.size  do 
with  j_frtWe[i.5tij(/cr [i.5can]]  do 

begin  (Reject  double  delinitions  by  going  to  done. loading.gr ammar  13); 

status  uncount.term;  i.scan  i.scan  +  1; 

end; 

write  {tty,  'Nonterminal  (s)»u');  iJine{true); 
while  t.scrtn  <  i.line.size  do 

begin  with  s.table[i.buffcr\i.scan\]  do 
begin  (Reject  double  definitions  by  going  to  done. loading. grammar  13); 
status  nonlcrm;  matrix  nil;  prods  nil;  appears.in  ♦—  nil; 
derives. empty  false;  precedtd.by  4—0;  followers  4— nil; 

end; 


i_scfln 


i.3can  +  1; 
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end; 

This  co(U*  is  used  in  section  Jl. 


13.  Rojort  redefinitions  of  a  character.  The  code  below  checks  the  previous  definition  of 
a  character  to  be  sure  that  it  is  not  being  redefined.  It  always  appears  in  the  context  of  a 
with  s buffer  [i^scan]]  do, 

(Reject  double  definitions  by  going  to  done. loading  ^grammar  13)  = 
if  status  ^  undefined  then 

begin  wrtteln(tty ,  Error i-6tij/er[i>scan],  'ulsualreadyudefined'); 
goto  done. loading. grammar ; 

end 

This  cade  is  used  in  sections  12,  12,  12,  and  12. 

14.  Ihe  production  portion  of  loading  a  grammar. 

Earh  lino  of  uiput  is  scanned.  The  program  expects  the  loft  hand  side  to  resemble  N 
followed  by  a  right  hand  side  consisting  of  production  pos.sibilities  separated  with  vertical 
bars.  The  box  operator  appears  as  an  up  arrow  in  the  input. 

(Get  productions  14)  = 
begin  while  true  do 

begin  writc(tty,  'Production»u');  Uine{false); 

if  (i-ft«j7‘cr[l]  =  'U')  A  {t.buff€r[2]  -  'P')  then  goto  done. getting. productions^ 

(Scan  the  left  hand  side;  goto  Jlush.production  if  it  has  bad  format  15) 

(Sean  the  right  hand  side;  goto  done. loading. grammar  if  it  has  bad  format  16) 
flush. production:  end;  ' 

done.getting.productions:  end 

This  code  Is  used  in  section  11. 

15.  The  left  band  side  should  consist  of  a  single  nonterminal  followed  by  an  arrow. 

(Scan  the  l.'ft  hand  side;  goto  fiush.pToduction  if  it  has  bad  format  15)  = 
left.sidc  i.buffer[[]; 
if  -^(cnpitalJetterlleft.side))  then 

begin  wrtt€ln[ttyy  Error :uproductionu8houldubeginuWithu2kjletter 

goto  flush.production; 

end; 

if  s. table [left. side]. status  •/  nonterm  then 
begin  writeln{tty ^  Error ty', /c/Lside,  'ushouldubeyaunonterminal'); 
goto  flush. production] 

end; 

if  i_6«^cr[2]  /  then 

begin  tvriteln{tty,  'Error:u-uexpectedualteru\/e/<.side);  goto  Jluah.production: 
end; 

This  code  is  uscfl  in  section  14. 
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16.  Tlie  riglit  hand  side  may  contain  several  production  possibilities  sepevrated  by  vertical 
bars.  The  code  below  rejects  small  letters  and  excessively  long  strings. 

(Scan  the  right  hand  side;  goto  doneJoading^grammar  if  it  has  bad  format  16)  = 
i.scan  *—  3;  jKn€w{p ^possibility); 
while  i.scan  <  iJine.size  do 
begin  case  LbuJJeT[i.3can]  of 
'I (Finish  a  production  possibility  19 }; 

(Box  the  previous  symbol  17); 
others:  begin  exit. char  ^  {.buffer [i^scan]; 
if  -^{capitalJetter (cur. char))  then 

begin  tvriteln{tty,  'Error tuunacceptableucharacteru',  cuf-cfcar); 

goto  done. loading. grammar; 

end; 

with  p. possibility^  do 
begin  size  ^  size  -f  1; 
if  size  >  max.prod.symbols  then 

begin  xvriteln(tty ,  'Error  lunoumoreuthaiiu',  max_prod.sym6o/s  :  0, 
'ulettersuinuauproductionupossibility'); 
goto  done  .loading. grammar ; 
end; 

( Include  cur.char  in  the  string  18 ) 
end; 
end 

end;  Lscan  i.scan  +  1; 
end; 

(Finish  a  production  possibility  19 ); 

This  code  is  used  in  section  14. 
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17.  An  ]  indicates  that  the  previous  symbol  is  boxed.  In  the  record  for  the  production 
possibility,  bid  is  set  the  the  location  of  the  boxed  character  in  tlie  stringy  «and  one  of 
jirst^hxd.  sccond.bxd ,  or  upectal^bxd  is  set  to  one,  depending  on  which  nonterminal  or 
terminal  character  is  boxed. 

{ Box  the  previous  symbol  17 }  = 
witli  p^pos.sibility  ]  do 
begin  bid  <—  size; 
if  size  —  0  then 

begin  'Error:unothingui8ubeinguboxed');  goto  Jlush.proiiuction; 

end; 

case  5-fuMe[5frinf;[.suc)].siafTi5  of 
novterrn:  case  sub.size  of 
1:  first ^bxd  ^  1; 

2:  second.bxd  ^  1; 
end; 

labelled:  speciaLbxd  ^  1; 

others:  begin  writeln(ity,  'Error :u',strmff [sue],  'ucarmotubeuboxed'); 
goto  done Joading ^grammar; 
end 
end; 
end 

This  code  is  used  in  section  16. 

18.  Here  is  the  code  that  adds  a  character  to  the  string  portion  of  a  production  possibility. 

If  the  character  is  a  nonterminal  then  it  is  also  added  to  the  sub.string.  The  instructions 
below  appear  within  the  scope  of  a  v/ith  p^possibxlity  do. 

(Include  cur^char  in  the  string  is)  £= 
string; [siec]  ^  cur. char; 
case  sJabl€\cur.char]. status  of 

labelled ,  count. term:  adjust[s.table  [cur  ^ckar]. index]  < —  adjust  [salable  [cur  .char],  index]-\-l; 
nonterm:  begin  sub.size  sub^size  +  1; 
if  sub.size  >  2  then 
begin  WTiteln{tty, 

'Error 
goto  doneJ 
end; 

su6_3trm|7[5iift_5i2e]  4-  cur^char; 

( Prepare  the  appe;iraucc  of  a  nonterminal  for  safe  order  computation  27 ) 

end; 

undefined:  u;n/c/n(//y,  'Error : u cur-cAar,  'uisuundeclared'); 
end; 

This  code  is  used  in  section  16. 


•uROiJnoreLjthanu2unonterminalSuinu8tuproduction(jpos8ibility '); 
oadiVi^.flrrammar; 
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19.  Finishing  off  a  production  possibility. 

The  scan  has  encountered  a  vertical  bar,  or  the  end  of  the  line  so  it  is  time  to  finish  a 
production  possibility.  There  are  two  cases: 

1)  The  production  possibility  is  free  of  nonterminals,  and  .^o  the  result  of  the  production 
can  be  entered  directly  in  the  matrix  associated  with  the  left  hand  side  nonterminal  at  the 
location  of  the  adjust  coordinates.  (This  part  of  the  procedure  will  be  considered  later.) 

2)  The  production  possibility  has  nonterminals  and  so  is  linked  into  the  prods  list  of  the 
left  hand  side  nonterminal. 

(Finish  a  production  possibility  19)  ^ 
begin  with  p .possibility  ]  do 

begin  (Prepare  the  completion  of  a  production  for  safe  order  computation  28); 
if  adjust  [1]  >  1  then 

begin  writ€ln(tty,  'Error ruonlyuoneu',  tndcx_sym6o/s[l], 

'ualloweduinuauproductionupossibility ');  goto  done  .loading  ^grammar; 

end; 

if  sub.size  —  0  then  ( Enter  a  string  of  all  terminals  55 ) 

else  begin  p.next  ^  sJablelleft. side]. prods;  sJable[left.side]. prods  +-  p. possibility; 
end; 

WTite(tiy,  *  Loaded  :ixj\  left. side  ^ 

for  I  1  to  size  do  writ€(tty^  siring [i],  *u'); 

writeln{tty,  'uwithuboxedupositionu',  :  0); 

end; 

p.new  (p.possibility); 

end 

This  code  is  in  sections  10  and  16. 

20.  Here  we  initialize  a  production  possibility  by  creating  a  new  p^right.side  and  setting 
the  appropriate  default  values. 

( Internal  gr.ammar  loading  procedures  20 )  = 
procedure  p.new{vzLr p. possibility  :  p.pir); 
var  i:  integer; 
begin  new (p. possibility); 
with  p.possibility  f  do 

begin  bxd  0;  speciaLbxd  ^  0;  first J>xd  0;  second.hxd  <—  0;  size  0; 

0; 

for  t  ♦—  1  to  max-co«nfa6/c_tcrmina/s  do  adjust  [t]  ^  0; 
end; 
end; 

This  code  is  used  in  section  11. 
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21.  Computation  of  the  safe  order. 

There  arc  two  kinds  of  lists  used  to  compute  the  safe  order.  An  a.list  contains  pointers 
to  productions,  and  a  cJist  contains  characters  representing  nonterminals. 

(Global  types  s)  +  = 
aJist  =  record  prod:  p.plr; 
a^next:  a.ptr 
end; 

cJist  =:  record  letter:  char; 
c^next:  c.ptr 
end; 

22.  The  safe.order^  once  it  is  computed,  is  available  throughout  the  program. 

(Global  variables  3)  *f= 

safe^order^  scan^safe^order:  c.ptr; 

23.  The  decrement.test.zero  function  subtracts  one  from  its  operand  and  returns  true 
when  the  result  is  zero. 

(Global  procedures  4)  += 

function  decrement  Jest. zero  (var  operand  :  integer):  boolean; 

begin  operand  operand  -  1;  decrement Jest^zero  (operand  =  0); 
end; 

24.  The  norm  function  sums  the  components  of  a  characteristic  vector. 

(Global  procedures  4}  += 

function  norm  {vector  :  characteristic. vector):  integer; 
var  index,  femp.norm:  integer; 
begin  temp.norm  0; 

for  index  to  number. indices  do  temp.norm  ^  iemp.norm  +  -jccf or  [index]; 

norm  femp^norm; 

end; 

25.  During  the  safe  order  compulation  a  depth  first  search  will  used  to  process  the  the 
nonterminals.  This  is  implemented  with  a  stack  called  unprocessed. 

(Local  variables  for  computing  the  safe  order  25}  = 
unprocessed  ^  new. unprocessed ^  new.follower:  e.ptr; 
new.appearance:  a.ptr; 
tail.of. safe. order :  c.pfr ; 
scan.s.table ^  being.processed:  char; 

This  code  is  used  in  section  11. 

26.  Initially  the  stack  and  safe  order  arc  empty. 

(Initialize  for  computing  the  safe  order  20 )  = 
unprocessed  ^  nil;  safe.order  ^  nil; 

This  code  is  uac<l  in  section  11. 
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27.  The  fragment  below  bthlds  a  reverse  directory  for  the  nonterminals,  it  appears  in 
the  code  that  is  loading  prodnetion  possibilities.  The  scan  has  encountered  a  nonterminal 
cur. char  and  already  loaded  cur. char  into  a  p.possibility ,  In  addition,  a  pointer  to  the 
p.possibility  is  linked  into  the  appear s.in  list  of  cur. char. 

(Prepare  the  appearance  of  a  nonterminal  for  safe  order  computation  27)  h 
new  {new.appearancc); 
with  new. appearance]  do 

begin  prod  ^  p.possibility;  a.next  *-  $Jable[cur. char],  appear  a  Jn; 
end; 

aJable[cur.char].  appears  .in  <—  new.appearance; 

This  code  is  uscil  in  section  18. 

28.  If  a  production  has  no  characters  that  are  countable,  and  is  free  of  nonterminals  that 
might  derive  countable  characters,  then  contributors  is  zero.  In  this  case  Icft.side  is  linked 
on  the  unprocessed  list  —  it  will  eventually  be  processed  and  marked  with  derives. empty  set 
true. 

(Prepare  the  completion  of  a  production  for  safe  order  computation  28)  = 
pJeft.side  left. side;  contributors  ^  norm(adjust)  + 
if  contributors  =  0  then 

begin  new  (new.unprocesaed); 
with  new.unprocessed]  do 

begin  letter  left.side;  c.next  unprocessed; 
end; 

unprocessed  new.unproceased; 
end; 

This  code  is  used  in  section  19. 
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29.  Tlic  s/ifo  onicr  is  fnr  roiiipiifinjj  iioiitcTiiiinal  iiwtriri'r.  Snpi)o^p  tlipro  is  a  prodnctioii 
^  ‘V  .1  arc  iKiiitorniiii.ils.  and  wv  arr  roinpiifiiij;  flic  [i,j.k)  entries  in  all 

the  matria-s.  then  the  entry  for  A  nnist  lie  onniiiifed  before  the  entry  for  N.  However, 
if  there  are  eomitable  t<-rininals.  like  A'  and  F.  ad.led  to  the  ri-ht  side  of  the  pro.lnrtion! 
N  -  XAF.  then  the  roniimtation  of  N  neeils  an  entry  from  A  with  a  smaller  norm.  If  wo 
compute  the  entries  in  the  order  of  .ascendiiif;  norm  it  is  no  longer  necessary  that  A  preccrlo 
N.  So  precedence  depends  on  the  prestmee  of  critical  ch.aracters. 

Precedence  is  furtiur  com])licated  by  productions  like  N  ->  AD,  where  N,  A,  and  D 
are  all  nontermm.als.  If  D  might  derive  a  string  without  conntable  characters  (a  pseudo 
empty  string),  tln-n  ,1  must  precede  N .  For  tins  rea-son  the  computation  of  the  safe  order 
is  .accomplished  in  two  ph.a.ses.  The  first  jih.asc  identifies  tho.se  nontermin.als  that  derive 

р. seudo  empty  strings.  Then  the  precedence  relations  are  apparent  and  the  .second  phase 

с. an  topologic.illy  .sort  the  nontermin.als  into  a  linear  order  that  is  consistent  with  these 
relations. 


(Compute  tlu  safe  order  29)  = 

(Mark  tiiose  nonterminals  (lorivin^  psendo  empty  strings  30 ); 

wrtfe(tl;i.  'Theufollowingunonterminalsucanuderiveupseudouemptyustrings lu'); 
for  scnn.sJfible  'A'  to  *Z'  do 
with  do 

if  (statris  -  rtonterm)  A  (krives.empty  then  WTite{tty ^  scan,$Aab{e)\ 
u>T\tcln[tiy^^  (Count  tlii  numlx'r  of  predecessors  for  each  nonterminal  31  )j 
(Topologically  sort  the  nonterminals  into  the  safe  order  32); 

WT\tc(ity.  'Theusafeuorderuisru');  sc  an. safe. order  ^  safe.order; 
while  scan. safe. order  /  nil  do 
with  scan. safe  .order  ]  do 

begin  write  (tty,  'u'  -  ktter);  sc  an. safe. order  ^  c.next: 
end: 

wrtt€ln(tty)\ 

This  code  is  ase<l  in  section  11. 
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30.  The  iiontcnninals  that  ran  derive  pseudo  empty  strings  arc  identifK'd  as  follows;  when 
a  grammar  is  loaded  any  prodnetion  free  of  ronntable  terminals  and  nonterminals  causes  its 
left  hand  sid<'  to  be  placed  on  the  unprocessed  stack.  The  loop  below  removes  thc.se  “empty 
derhers  from  the  stack  and,  using  the  appearsJn  lists,  linds  all  occurrences  of  the  empty 
derivers  in  other  productions  and  diminishes  the  contributors  fields  of  these  productions. 
When  a  contributors  field  is  reduced  to  zero  another  empty  deriver  has  been  found,  so  it  is 
puslu'd  on  tlic  unprocessed  stark. 

{Mark  thoso  iiontrrininals  (leriving  pscMido  empty  strings  30)  = 
while  mipToressed  ^  nil  do 

begin  being. processed  ^  unprocessed] .letter;  unprocessed  unprocessed^. cMxt; 
with  sJahlc  [being ^processed]  do 
begin  derives.cmptrj  true; 
while  appears.in  /  nil  do 
with  appears.in] .prod]  do 
begin  if  decrement  Jest  ..zero  {contributors)  then 
begin  ncw(new.unprocessed); 
with  newMnprocessed]  do 
begin  letter  pJeft^stde;  c.next  ♦—  unprocessed; 
end; 

unprocessed  4—  new  ^unprocessed; 
end; 

appear  s.in  ^  appears  An],  a.next; 

end; 

end; 

end 

This  code  is  used  in  section  20. 
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31,  Once  tho  empty  derivers  arc  ideiitiPicd  wc  can  find  all  tlie  precedence  relations.  If  A 
precedes  B  then  one  is  added  to  the  prcccdcd.hy  field  of  B  and  B  is  linked  in  the  Jollov^trs 
list  of  id. 

define  add. scan  Jo  (^)  = 

begin  ucw[n€W.follower)\ 
with  new  .follower]  do  • 

begin  letter  ^  scan.sJable;  C-next  ^  $Jable[tt]. followers; 
end; 

sJable\4]. followers  newjollower;  preceded.by  ♦—  preceded J>y  +  1; 

end 

(Count  the  number  of  predecessors  for  each  nonterminal  31 )  = 
for  scan.sJable  'A'  to  'Z'  do 
begin  with  sJahle  [scan.sJable]  do 
if  status  =  nonterm  then 
begin  p.possibiUty  ^  prods; 
while  p.possibiUty  ^  nil  do 
with  p.possibiUty]  do 

begin  if  norm{adju3t)  =  0  then 
case  sub.size  of 
1:  add_sran.io(sti5.strin^[l]); 

2:  begin  if  s.table\sub.string[i\].derives.empty  then 
add.scan  Jo  (substring  [2]); 

if  sJable[sub.strivg[2]].deriv€s.empty  then  odd_scanJo(s«A.3trfny{l]); 
end 
end; 

p.possibiUty  p.possibiUty ] .p. next; 

end; 

end; 

end 

This  code  is  used  in  section  20. 

32.  The  topological  sort  starts  wiiii  a  scan  of  tho  symbol  table,  in  order  to  find  nonter¬ 
minals  with  zero  preceded.by  fields  that  arc  ready  to  be  placed  in  the  safe  order. 

( Topologiccilly  sort  the  noiitenninals  into  the  safe  order  32 }  = 
for  scan.sJable  'A'  to  'Z'  do 
with  s J abl e  [s c an. s. table]  do 
if  status  =  nonterm  then 

if  preceded.by  =  0  then  put  Jn.safe. order  (scan.sJable); 

(Clu?ck  that  fill  iiontcrinin<*ds  arc  in  the  safe  order  34); 

This  co<lc  is  used  in  section  29. 
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33.  When  a  nontcrnnna!  is  moved  to  the  safe  order  the  preceded. by  fields  of  each  of  its 
followers  arc  diminished  by  one.  If  any  of  these  fields  reach  zero  more  items  are  added  to 
the  safe  order. 

(Safe  order  procedure  33)  = 
procedure  pxitArxMft^ordet {now. safe  :  cAor); 
var  new. safe. order:  e.pit] 

begin  new [new.safe. order)]  new. safe. order \Aeiter  ♦-  now.safe] 
if  safe.order  =  nil  then  safe.order  ^  new. safe. order 
else  iail.of .safe.order ] .c. next  new. safe. order] 

iail.of.safe.ordtr  ^  new. safe.order]  tail. of. safe. or der^,e. next  ^  nil; 
with  s. table  [no w.safe]  do 

begin  preceded.by  < - 1; 

while  followers  /  nil  do 
with  followers]  do 

begin  if  decrement.test.2ero{sJable[letter].preceded.by)  then 
put.in.safe. order  {letter)] 
followers  c.next] 
end; 

end; 

end; 

This  code  is  used  in  section  11. 


34. 

( Check  that  all  nonterminals  are  in  the  safe  order  34 )  = 
for  scan.s.table  'A'  to  'Z'  do 
with  s.table[scan.s.table]  do 
begin  if  status  =  nonterm  then 
if  preceded.by  “1  then 

begin  writeln{tty^  'Error luthereumustubeuaucycleuinutheugrammar'); 

goto  done. loading. grammar] 

end; 

end 


This  code  is  used  in  section  32. 
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35.  Matrix  Data  Structure  and  Procedures. 

TIk’  (lata  ^'tnu•tur('  and  code  t!;at  follows  iinph'nn'nts  inafric(\s  w’ith  a  varialdc  num!)rr 
(^f  dinicnsi  ni<.  TIu'H'  arc  number .ir,ilirf.s  dinu'nsions  all  of  size  maiJotnl  -f  1.  Each 
n(n\t('rniinal  will  have  one  of  tliese  inatrirt\s  roidaimiii;  pointers  to  the  walk  strnrturc. 

A  matrix  consists  of  a  seri(\^  of  one  dinu'nsional  axes.  The  lower  dimensional  (interior) 
«axes  contain  ])oinlers  to  otluT  axes;  only  the  hi;;lK\st  dinu'nsional  (exterior)  axes  hold  the 
ccnitf'nts  of  the  matrix. 

Normally  aecc'ss  to  the  matrix  re(|nires  a  pointer  dc'refercnce  for  each  dinumsion.  To  avoid 
this,  the  array  is  a(ldr('ssed  with  a  rn^coord  that  has  extra  fields  to  retain  tlicse  pointers. 
If  the  same  locatic^n  is  acet'ssed,  or  if  a  change  is  made  only  in  the  highest  dimension  then 
the  rn^coord  will  still  ludd  the  relevant  pointers. 

An  rn.ptr  is  a  pointer  to  an  niMzis. 

(Global  types  s)  -f - 

m.aiisJype  --  (m/erfor,  exterior); 
m_axi.s  array  [0  ..  maxJoial]  of 
record  case  rn.axisJype  of 
interwr:  (rieitJevel  :  m.ptr); 
exterior:  (entry  :  w^ptr) 
end; 

rn^coord  =  record  symbol:  char;  {This  m^coord  accesses  the  matrix  for  nonterminal 
symbol  } 

tnt:  arr Ry  [{,.  max ^conntabltJerminals]  o(  inte^^er;  {  Integer  coordinates  } 
pfr:  array  [1  mnx.countcble^tcrminals]  of  m^ptr  (Pointer  coordinates} 
end; 

36.  Allocation  and  initialization  of  matrix  axes. 

(Matrix  procedures  30 )  = 

procedure  m_neti?_axi,s(i  :  integer;  vara  :  m.pfr); 
var  j:  integer;  w\  w.ptr; 
begin  new  (a); 

if  t  =  number. indices  then  (This  is  the  highest  dimension.  } 
for  j  0  to  maxJotal  do 
begin  new (w,  trivial); 
with  w]  do 

begin  value  ^  0;  w.next  4—  nil; 
end; 

a][j]. entry  <-  w; 

end 

else  {  This  is  an  interior  dimension.  } 

for  j  +~-  0  to  max.total  do  (i]{j].next .level  nil; 

end; 

See  also  sections  37,  40,  and  41. 

This  code  is  nsc<l  in  section  1, 
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37.  This  is  a  preliminary  access  procedure.  It  allocates  uiiprohcd  axes  if  necessary, 
initializes  them,  and  stores  helpful  pointers  in  the  m.coord  record.  The  m.locate  procedure 
must  be  applied  to  an  m.coord  before  any  m.access  operations. 

(Matrix  procedures  36} 
procedure  mJocate(\arm  :  m.coord); 
var  i:  integer; 
begin  with  m  do 

with  sJable[symbol]  do 

begin  if  matrix  =  nil  then  m_neu;.axt5(l,  matrix); 

ptr[l]  matrix; 

for  i  2  to  number  Jndicea  do 

begin  if  ptr[i  -  l]t[int[i  l]].nextJevel  =  nil  then 
m,ncty>axM(i, ptr[i  -  l]t[int[i  —  l]].nextJevel]; 
ptr[i]  4—  ptr[i  -  l]|[iat[i  --  i]].nextJevel; 
end; 
end; 

end; 

38.  Accessing  the  entries  of  a  matrix. 

We  assume  that  mJocate  has  been  applied,  so  that  ioc  contains  the  proper  pointers. 
We  then  need  only  use  ptr[num6er-indiccs]  to  find  the  highest  dimensional  axis,  and 
int[ntim6cr-indicc3]  to  find  the  proper  entry  in  this  axis. 

define  m>occcs5(#)  =  It, ptr  [number  Jndice3]'\[it. int  [number  Jndices]]. entry 

39.  To  keep  track  of  which  characteristic  vectors  have  been  computed,  there  is  a  global 
matrix  of  booleans  called  done  ^already .  The  next  few  modules  implement  this  boolccui 
matrix  in  a  manner  similar  to  the  preceding  modules. 

(Global  types  s)  += 
m^b^ptr  =  tm-6.axij; 

m_6.axi3  =  array  [0  maxJotai]  of  record  case  rn.axiaJype  of 
interior:  [nextJevel  :  m^b^ptr); 
exterior:  {entry  :  boolean) 
end; 

40.  Allocation  and  initialization  of  the  boolean  matrix  axes. 

(M<atrix  procedures  36) 

procedure  m-6_neu;-axM(t :  integer;  vara  :  m.ft.ptr); 
var  j:  integer; 
begin  neu;(a); 

if  i  —  number  Jndices  ti.en  {  This  is  the  highest  dimension.  } 
for  j  4—  0  to  max. total  do  a][j]. entry  4—  false 
else  {  This  is  an  interior  dimension.  } 
for  j  4—  0  to  max.total  do  a][j].nextJev€l  4—  nil; 
end; 
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41.  Preparing  to  access  the  boolenn  matrix. 

(Matrix  procedures  36)  += 

procedure  m^bJocate(yaTm  :  m.coord); 
var  i:  integer; 
begin  with  m  do 

begin  if  done^already  =  nil  then 

begin  m.b.n€w.axis(l,  done.alftady);  6.pfr5[l]  ^  done^dready; 
end; 

for  t  2  to  number  .indicts  do 

begin  if  b.ptrs[i  -  l]t[mt[i  -  l]\.nextJevel  =  nil  then 
Tn.b.n€w.axis(i.  b.pirs\i  -  l]T[mt[»  -  l]].nextJci;c/); 
fr_p<r5[t]  ^  6_ptr5[»  -  l]T[mt(i  -  l\],n€xiJtvel; 
end; 
end; 
end; 

42.  Accessing  the  entries  of  the  boolean  matrix. 

define  m-6_acce5.?(#)  =  b.ptrs[number.indices]^U.int[number .indices]]. entry 

43.  The  boolean  matrix  is  a  global  variable.  There  is  also  a  global  array  of  pointers  to 

access  the  matrix. 

(Global  variables  3)  += 

done.already:  m.b.ptr; 

b.ptrs:  array  [1  ..  max.countable.terminds]  of  m_6_ptr; 
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44.  Selector  Walk  Structure  and  Construction  Procedures. 

The  walk  structure  is  built  by  count  using  the  procedures  w.multiplyy  u;.5um,  w^build 
and  w.single. multiply ,  It  is  used  by  toalk  to  quickly  generfate  a  string  specified  by  an  integer 
selector.  The  total  number  of  strings  derivable  from  a  node  of  the  walk  structure  is  recorded 
in  tlie  value  field  of  the  w.node.  When  walking  the  structure,  selector  should  aways  be  in 
the  range  0  ..  value  ~  1.  There  are  two  types  of  nodes,  trivial  and  drastic.  The  trivial 
nodes  have  left  and  right  sons.  They  are  formed  by  the  summing  process,  ajid  so  the  value 
field  is  the  sum  of  the  values  of  the  two  children.  The  walk  procedure  will  turn  either  left 
or  right  at  a  trivial  node.  The  drastic  nodes  are  the  result  of  the  multiplication  process. 
At  a  drastic  node  the  walk  procedure  must  (in  an  intertwined  order): 

1)  Output  the  terminal  symbols  in  the  production  used,  p.usedy  and 

2)  Walk  the  structures  for  both  the  nonterminals,  firs^walk  and  seco^walk, 

A  w.ptr  is  a  pointer  to  a  w.node.  The  w^next  field  is  used  to  link  together  nodes  that 
have  been  summed  until  they  are  built  into  a  balanced  structure. 

define  t£;_va/uc(#)  =  #|.va/tic 

(Global  types  s)  -t-= 

wAype  —  (trivial y  drastic)] 
w,node  =  record  value:  integer] 
w.next:  w.ptr] 
case  state  :  w.type  of 
trivial:  (leftyright  :  ta^pfr); 

drastic:  (split.factor  :  integer]  p.used  :  p-pfr;  fir s. specials y  seco. specials  :  integer] 
firs.walkyseco.walk  :  w.ptr) 


end; 
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45.  Tile  multiply  prorcduro 

The  w. multiply  j)rocc(lnro  crwRos  a  draattc  w.node.  Tlic  Viduc  field  is  essentially  the 
produrt  of  tlio  two  value  fields  of  the  nonteriiiinals  in  p.nsed,  with  tfie  countable  symbols 
divided  areordinR  to  first. loc  and  secondJoc.  Tliis  product  is  modified  by  splitjactor 
becatise  of  the  exponential  nature  of  the  series  in  the  sjuxial  labelled  character.  For  example, 
suppose  X  is  the  labelled  character,  and  we  are  multiplying 


then  a  typical  term  of  tho  product  will  be 


The  spht.factor  plays  tho  rolo  of  tho  binomial  coefficient,  although  below  we  are  using  the 
multinomial  coefficients  because  tliere  are  really  three  contributions  to  the  product:  the 
number  of  X  s  in  the  production  itself,  adjust  [i];  the  number  of  X’s  in  the  first  nonterminal, 
Jirs. specials;  and  the  number  of  A"*s  in  the  second  nonterminal,  seco.specials.  When  the 
walk  procedure  traverses  this  w.node  it  will  have  a  list  of  labels  that  it  intends  to  assign 
to  terminal  As.  The  spUt.factor  counts  the  number  of  ways  of  dividing  the  list  into  three 
parts.  If  any  of  the  three  parts  are  ‘‘boxed”  then  1  is  subtracted  from  the  corresponding 
index.  Algebraically,  this  is  a  consequence  of  the  shifting  of  tho  exponential  scries  with  the 
differentiation  and  integration  operators.  Combinatorially,  this  corresponds  to  fixing  the 
location  of  the  smallest  Label  in  the  separation  process. 

(Walk  structure  builders  45)  = 

procedure  v}.mnltiply{var product  :  w.ptr;  p.possibility  :  p-ptr; 
first  Joe ,  secojidJoc  :  m.coord); 
begin  with  product  ]^p.posfiibility]  do 

begin  p.used  p. possibility;  firs.specials  ^  firstJocAnt[i]\ 

seco.specials  ^  sccond_/or.mf  [l];  spUt.factor  <-  mu/t [adf/ust [1]  -  apeciaLbxd, 
firs.specials  -  first.hxd ,  seco.specials  -  second^bxd]; 
firs.walk  <-  m. access  [Jirst.loc);  seco.walk  m.access  (secondJoc); 
value  split. factor  *  w.value  (firs.walk)  ♦  w. value  (seco.walk);  w.next  ^  nil; 

end; 
end; 

Sec  also  sections  40,  47,  and  48. 

This  code  is  use<l  in  section  1. 
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46.  The  multiply  procedure  with  a  single  operand. 

The  w^singlc ^multiply  routine  is  identical  to  w^multiply  except  that  the  production  has 
only  one  nonterminal. 

(Walk  structure  builders  45)  += 

procedure  w.singlc ^multiply  {var  product  :  w^ptr  \  p^posaibility  :  p^ptr\  firsUoc  :  m.coord); 
begin  with  product  ^^p^possibility]  do 
begin  paused  ^  p^possibility;  firs^sptcials  ♦-  first  Joe. int[l]; 
split Jactor  *—  mu/t(adtytist[lj  ~  speciaLbxd.^  firs. specials  -  firsLbxd^O]; 
firs.walk  m.access {first Joe);  value  spliLfactor  *  w.value{firs.walk); 

seco.specials  0;  w.next  ^  nil; 
end; 
end; 

47.  The  sum  procedure. 

The  eventual  purpose  of  the  w.sum  procedure  is  to  produce  a  w.node  with  two  descen- 
dents,  left  and  rights  and  a  value  field  equal  to  the  sum  of  the  values  of  the  two  dcscendents. 
However,  in  order  to  optimize  the  data  structure  for  later  walks,  the  w.nodes  are  first  linked 
together  by  their  w.next  pointers  in  an  accumulator  list.  Later  on,  the  w.build  procedure 
will  create  a  tree, 

( Walk  structure  builders  45 )  += 

procedure  tw-sum (var  occti mw/af or  :  w.ptr;  new  :  w.ptr); 
begin  new^.w.next  accumulator;  accumulai.:  ^  new; 
end; 

48.  A  balanced  tree  builder. 

This  procedure  builds  a  balanced  tree  from  a  list  by  repeatedly  combining  the  first  two 
nodes  and  reinserting  them  at  the  end  of  the  list. 

{Walk  structure  builders  45)  += 
procedure  to>6uiW  (var  accumu/ator  ;  w.ptr); 
var  tail  ^joined:  w.ptr; 
begin  tail  ^  accumulator; 

while  {lail^.w.next  ^  nil)  do  tail  ^  tail^.w.next;  {Find  the  end  of  the  list } 
while  accumulator  ],w.next  ^  nil  do 
begin  new  {joined ^  trivial);  joined]  Jeft  ♦-  occumu/otor ; 
joined]. right  ^  accumulator ].w.nexi; 

joined].value  <—  accumulator]. value  +  accumulator ].w.next]. value; 
tail].w.next  joined;  tail  ^joined;  tail].w.n€xt  nil; 
accumulator  ^  accumulator ].w.next].w.next; 

end; 

end; 
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49.  Counting  Pi  ?cedurcs. 

The  procedure  count  is  called  with  a  loc  parameter  that  coiitiuns  a  nonterminal  and  a 
cliaracteristic  vector,  indicating  that  tlie  user  desires  an  accounting  of  the  derivations  that 
start  with  Ihe  nontejiuiiial  and  have  a  final  composition  equal  to  the  vector. 

If  loc  is  not  alrcc  dy  computed,  the  code  below  builds  up  a  solution  to  loc  by  first 
computing  all  vectors  with  norm  less  than  loc.  In  tlie  loop  bedow  total  ascends  through  norm 
sizes.  For  a  fixed  value  of  total  the  inner  loop  distributes  total  among  the  components  of  the 
characteristic  vector  taryetJoc,  subject  to  the  constraint  that  no  component  can  exceed  its 
corresponding  component  in  loc.  Thus  we  generate  cill  taryet.loc  s  that  are  componentwise 
less  than  loc,  in  ascending  order  of  their  norms. 

define  ncxtJotal  —  10 
define  next.production  =  11 
(Counting  procedures  49)  = 
function  count  (loc  :  m^coord):  integer; 
label  next. totals  next.production; 

var  par  it  nl. sums:  array  [O  ..  max.count  able. terminals]  of  integer; 
chunk ,  I ,  col.l ,  coL^^ .  total ,  sub.total :  integer ; 
targetJoc,  delta  Joe  Jirst  Joe ,  second.loc:  m.coord; 
scan.productions:  p.ptr; 
accumulator ,  new. drastic:  w.ptr; 

begin  m.b.locat€{loc);  {Check  donc.already  to  see  if  loc  needs  computing} 
if  -^m.b. access  (loc)  then 

begin  new  (ncw.drastic ,  drastic);  new. drastic  ].w.next  nil; 

sub  jotal  0;  ;;arh'a/_5«m5  [O]  0; 

for  coLl  1  to  number. indices  do 

begin  subJotal  sub.total  loc.tnt[coLl];  partial. sums [col.l]  sub.total; 

taTget.loc.int[coLl]  0; 

end; 

for  total  +—  0  to  partiaLsums[n7imber. indices]  do 
begin  sub.total  ♦—  total;  col.l  number. indices; 
while  true  do 

begin  while  sub.U  tal  >  0  do  {  Disperse  sub.total  leftwards  } 

begin  chunk  ♦—  min(sub. total ,  loc .int[col.l]);  targct.loc  Jnt[col.l]  chunk; 

sub.total  sjibJotal  —  chunk;  col.l  col.l  -  1; 

end; 

(Compute  all  entries  at  targct.loc  50 ) 
if  sub.total  —  total  then  goto  next.total; 

while  (targct.loc .t7it[col.l  -|-  l|  =  0)  V  (sub.total  =  partial.sums[col.l])  do 
{  Scan  rightwards  to  find  a  column  to  diminish  } 
begin  col.l  col.l  +  1;  sub.total  <—  sub.total  +  target.loc .int[coLl]; 
target. loc. int [col.l]  ^  0; 
if  sub.total  =  total  then  goto  next.total; 
end; 

target.locJnt  [col.l  +1]  +-  target. loc, int  [col.l  +  1]  -  1;  sub.total  sub.total  -h  1; 
end; 

next.total:  end; 
end; 
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count  ♦-  u;.vaiuc(m-a(rc«55(/oc)); 
end; 

Tbifl  code  is  used  in  section  1. 

50.  For  a  particular  target  Joe,  the  code  below  scans  the  nonterminals  in  safe^order,  and 

for  each  nonterminal  it  scans  the  production  possibilities. 

( Compute  all  entries  at  target  Joe  50 )  = 

m_6Jocafc(tar^cL/oc);  {Check  doneMready  to  see  if  target  Joe  needs  computing} 
if  -«m-6-acccs5(iar^cL/oc)  then 

; 

begin  m.b^aecess (target Joe)  ^  true;  scan^safe^order  +-  safe.order; 
while  scan_5a/c-or(/«r  ^  nil  do 

with  sean.aafe. order]  do 

;  J 

begin  target  Joe.  aymbol  ^letter;  mJocate  (target  Joe); 

>  i 

aceumulator  ^  m.acceas (target Joe);  scan-productiona  *-  sJable[letter].proda; 
while  scan.productions  /  nil  do 

with  scon-productf ons  I  do 

% 

begin  ( Compute  the  effect  of  a  particular  production  51 ); 

1 

'  i  .1 

scan^productions  p^next; 

i- 

'  ■'s- 

: 

end; 

'  i 

u;.5tn7cf(accumtt/(jfor);  m.aeeess (target Joe)  ^  aceumulator; 

acan^safe^order  ^  c^next; 

end; 

r.i 

^  % 

end; 

1 
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Thi<)  code  is  used  in  section  40. 
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51.  At  this  point  wc  arc  ready  to  compute  the  contribution  of  a  single  production. 
Suppose  the  i)roduction  is 

XFA^GBGH 

and  wc  are  counting  derivations  that  start  with  N  and  have  final  composition  (6,5,2).  The 
code  below  subtracts  the  adjust  vector  contained  in  the  production  possibility,  (1,1,2), 
obtaining  a  delta  Joe  vector  of  (5,4,2).  If  the  ^^possibility  has  only  one  nonterminal,  then 
all  of  (5,4,2)  must  be  contributed  by  this  nonterminal.  Here,  however,  there  arc  two 
nonterminals,  so  (5,4,2)  is  divided  in  all  possible  ways,  as  described  in  the  next  section. 

(Compute  the  effect  of  a  particular  production  51 )  = 
with  tanjeiAoc  do 

begin  {  Subtract  the  counts  of  characters  for  this  scnn.pfoduciions  from  the  totals  } 
for  *  1  to  number Andices  do 

begin  delta  doc  Ant  \i]  mi[i|  -  adjust[i]; 

if  delta  Joe  Ant  [i]  <  0  then 

goto  next.productton  {  There  is  no  way  to  use  this  possibility  } 
end; 

delta  Joe. symbol  ^  5«6>s/nnf7[l];  mJocate  {delta  Joe); 

case  sub.size  of  {  0;  p.possibilities  without  nonterminals  are  entered  directly  into 
the  matrix  } 

1.  begin  u).stngle.multiply{n€w.drasttc^sean.production3y  deltadoc)\ 

(If  new^drastxc  is  nonzero  then  add  it  to  the  accumulator  53)* 
end; 

2:  {Compute  with  two  nonterminals  52); 

end; 

next^production:  end; 

This  code  is  used  in  section  50. 
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52.  Computinj;  the  contribution  of  a  production  possibility  witli  two  nonterminals. 

The  example  of  the  preceding  section  has  two  nonterminals,  A  and  B,  in  the  production 

possibility  that  together  must  contribute  (5,4,2)  characters  to  the  derived  string.  The 
code  below  divides  delta  Joe  (which  is  (5,4,2)  in  this  example)  into  first  Joe  and  seeondJoc 
by  a  cotinting  mechanism.  For  each  partition,  the  number  of  strings  derived  from  A  is 
multiplied  by  the  number  of  strings  derived  from  B,  and  the  results  of  all  partitions  are 
summed  together.  Multiplication  cuid  summation  are  performed  by  w^multiply  and  iw.sum, 
which  are  clocely  related  to  ordinary  multiplication  and  addition,  but  take  into  account  the 
exponential  nature  of  the  generating  function  and  the  construction  of  the  walk  structure. 
The  code  below  is  the  inner  loop  of  the  program. 

(Compute  with  two  nonterminals  52)  = 
begin  first  Joe  delta  Joe;  seeondJoc  .symbol  su5,3<rfng[2]; 

for  i  1  to  number  .indices  do  second  Joe. int[i]  ^  0; 

mJocate  (second  Joe);  w. multiply  (new.drastic  ^  scan.productions ,  first  Joe  ^  second  Joe); 

(If  new.drastic  is  nonzero  then  add  it  to  the  accumulator  53); 
coL2  number  .indices; 
while  coL2  >  0  do 

begin  fiist.loc.int\eol.2]  first  Joe  Ani[coL2]  -  1; 

if  first  Joe  .int[col.2]  <  0  then 

begin  first.loc.int[coL2]  delta  Joe. int[coL2];  second  Joe. int[coL2]  ^  0; 

coL2  col.2  ~  1; 

end 

else  Degin  second. loc  .int[col.2\  ^  second  Joe  .int[coL2\  +  1; 
if  col.2  <  number. indices  then 

begin  m.locate (first Joe);  mJocate(second.loc); 
end; 

w.multiply  ( new.drastic ,  scan. productions ,  first  Joe ,  second.loc ); 

(If  new.drastic  is  nonzero  then  add  it  to  the  accumulator  53); 
col.2  ^  number. indices; 

end; 

end; 

end 

This  code  is  used  in  section  51. 

53.  Relevant  new.drastic  nodes  are  added  to  the  walk  structure. 

(If  new.drastic  is  nonzero  then  add  it  to  the  accumulator  53)  = 
if  new.drastic] .value  ^  0  then 

begin  w.sum (accumulator ,  new.drastic);  neto (new.drastic ^  drastic); 

nev).drastic].w.next  nil; 

end 

This  code  is  used  in  sections  51,  52,  and  52. 
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54.  Adtleiuluni  to  the  i^rair.inar  loading  code.  Here  we  define  variables  for  the  next 
module. 

{Local  variables  for  loading  matrix  entries  54)  = 
loc\  m,coord\ 

new^drastic ^  already:  w^ptr; 

This  code  is  used  in  section  11. 

5a.  VViien  a  production  is  free  of  nonterminals,  it  is  entered  directly  into  the  entry  of 
thr  matrix  for  the  nonterminal  on  the  left  side  of  the  i)roduction.  The  code  below  fits  in 
tne  context  of  a  with  p^possibility  do.  The  adjust  array  of  the  p.possibility  describes  the 
makeup  of  the  production;  so  it  is  used  to  address  the  matrix. 

(Enter  a  string  of  all  terminals  55)  = 
with  loc  do 

begin  symbol  i-  lefLside; 

for  t  1  to  nTzm6er.tndices  do  irdfi]  f~  adjust  [i]; 
mJocate(loc);  new  (jiew.drastic ^  drastic); 
with  new^drastic  f  do 

bcgjn  value  *—  1;  w^next  ^  ml;  split-factor  1;  p-used  ^  p-possi5i7ity; 

firs-specinls  0;  scco.spccta/s  0; 

end; 

already  ^  m-access{loc);  wsum (already ,new-drastic)]  m.acce3s(toc)  4-  already, 
end 

This  code  is  usat  in  section  19. 


56.  Label  Lists. 

Now  we  have  coini)lctr(l  tlic  procedures  that  build  the  data  structures,  and  wo  are  ready 
to  use  them.  The  top  h'vcl  call  to  the  walk  procedure  begins  with  a  list  of  the  integers 
1  . .  rj.  Subsequent  calls  to  walk  will  have  fragments  of  this  list,  the  fragmentation  being 
performed  by  the  split  procedure.  Eventually  each  integer  in  the  list  will  become  a  label 
for  one  of  the  special  chiiractors. 

{Global  types  s)  += 

Lptr  —  ]Lltst;  l.list  =  record  lab:  integer] 

Lnext:  Lptr; 
end; 

57.  Append  a  label  to  the  end  of  the  list,  using  the  pointer  Lend  to  quickly  find  the  last 
item  of  the  list. 

{ Global  procedures  4 )  += 

procedure  Lappend (yar  LbeginJ-ends  new  :  Lptr); 
begin  if  Lbegtn  =  nil  then  Lhegxn  4—  new 
else  Lend Lnext  4—  netu; 
nevjJ.Lnext  4-  nil;  Lend  4—  new: 
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58.  Procedures  to  Walk  and  Produce  a  String. 

According  to  the  value  of  selector^  the  procedure  split  separates  a  list  of  labels,  t,  into 
three  lists,  oi  ,  e-?,  and  oS  with  sizes  ni ,  n5,  and  nS,  The  multinomial  coefficients  govern 
the  splitting  process. 

{Walking  procedures  58)  = 

procedure  split(i :  Lptr;  selector  ^nl  yn2  ^nS  :  integer;  var  of,  :  Lptr); 

var  cur^ol^end,o2.end^oS.end:  Lptr; 
begin  ol  ^  nil;  o2  ^  nil;  oS  ^  nil; 
while  i  ^  nil  do 

begin  cur  ♦-  i;  i  i],Lnext; 
if  selector  <  Tnult\nl  ~  I,n2,n5]  then 

begin /.append (oi ,  cur);  nl  ni  —  1; 

end 

else  begin  selector  selector  -  mult[nl  -  I,n2,n5); 
if  selector  <  mult[nl^n2  -  l,n5]  then 
begin  Lappend(o2 ^  o2^€nd,  cur);  n2  ^  n2  -  1; 

end 

else  begin  selector  ^  selector  -  mu//[ni,n2  -  l,n5); 
if  selector  <  mult[nl  ^n2 ^nS  —  1)  then 
begin  Lap  pend  (oS  ^oS^end,  cur);  nS  ^  nS  --  1; 
end 

else  teri7c(//y,  'Error : uselectoruinappropriateuinusplituprocedure . '); 
end; 
end; 
end; 
end; 

See  also  section  59. 

This  code  is  used  in  section  1. 
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59.  Tiir  nctunl  \va!kiii<;  procedure. 

The  xralk  procedure  traverses  a  data  structure  that  is  prepared  by  other  portions  of  the 
progr<itn  (counf ,  w. multiply  ^  . . .).  The  purpose  ot  malk  is  to  generate  a  string  fron^ 

the  grammar;  srUctor  specili<'s  the  string  to  be  generated,  and  fahr.h  is  a  list  of  integers 
that  are  to  be  attaclu'd  to  the  occurrences  of  one  particular  t(Tininal  symbol. 

The  vjdH  procedure  first  dispenses  with  the  trivial  nodes  by  branching  left,  except  when 
selector  is  larger  than  the  value  of  the  left  son,  in  which  case  the  left  value  is  subtracted 
from  selector  and  tfie  right  branch  is  taken. 

EventUtally  a  drastic  node  is  reached -- these  arc  processed  in  the  next  module. 

(Walking  procedures  58) 

procedure  walk(cur  :  w.ptr;  selector  :  inteyer;  labels  :  Lptr); 

\ar  share,  so  Jar  ,i:  integer;  the  Mx ,  spec. label ,  firs. labels  ,seco  .labels:  Lpir; 
begin  while  cur  state  “  trivia/  do 
if  selector  <  cur ] Jeft] .value  then  cur  ^  cur]. left 
else  begin  selector  ^  selector  -  cur  ]. left],  value;  cur  cur],right; 
end; 

with  cur  I,  c«r  f.p.u.sedt  do 

if  value  >  0  then  ('lYavorso  a  drastic  node  60 ) 
end; 

60,  A  drastic  node  indicates  that  a  production  is  to  bo  applied,  so  the  procedure  must 
divide  the  lalxds  between  the  special  character  and  the  nonterminals  in  the  production. 
Part  of  selector  is  removed  (by  mixed-radix  arithmetic)  and  used  to  govern  the  division 
process.  If  anything  is  boxed  then  the  smallest  label  is  stripped  from  the  beginning  of  the 
list,  saved  in  the.box.  and  then  returned  to  the  appropriate  list  after  the  split  procedure 
divides  the  labels. 

(Traverse  a  drastic  node  60 }  = 
begin  if  bxd  >  0  then  {  Remove  the  smallest  label } 
begin  the. box  ^  labels;  labels  labels ].Lnext; 

end; 

ssplit  (labels,  selector  mod  split  Jactor ,  adjust  [l]  -  speciaLbxd ,  firs. specials  -  first. bxd, 
seco. specials  -  second.bxd,  spec. label ,  firs. lab  els ,  seco. labels); 
selector  selector  div  split.factor; 

(Put  the  smallest  label  into  the  appropriate  list  61 ); 

(Output  the  selected  string  62); 
end; 

This  code  is  used  in  section  59. 
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61,  The  sniallt'i^t  label,  the^boXy  is  linked  at  the  begining  of  the  list  corrcspondL*5g  to  the 
box  operator  in  the  production. 

(Put  the  smallest  label  into  the  appropriavc  list  61 )  = 
if  jspccial.bid  >  0  then 

begin  thc.box  f.lMfxt  nil;  specJabel  ^  the^box\ 
end; 

if  first.bxd  >  0  then 

begin  the.box]A.n€xt  ^  firsJabels;  firsJabels  the^box; 
end; 

if  sccond^bid  >  0  then 

begin  the.bQx^.Lnext  itcoJabeh;  secoJabds  ♦—  the^box; 
end; 

Th  is  code  is  used  in  section  60. 

G2.  With  the  Ial)els  ready,  the  code  below  scans  the  production,  outputting  terminals  and 
calling  itself  recursively  for  the  nonterminals.  The  original  selector  is  divided  into  three 
parts,  one  part  to  control  the  split  procedure  (as  we  saw  above)  and  two  parts  to  give  to 
the  recursive  calls  on  the  nonterminals  in  the  production. 

(Output  the  selected  string  62)  H 
so  Jar  0; 
for  t  ^  1  to  size  do 

case  s_<a6/e[strin/7[j]].strttus  of 

uncount -term,  count Jerm:  if  string/ [t]  'E'  then  tyritc(ffy, string [t],  5 

labelled:  begin  tvrite(tty^string[i],  spec. label], lab  :  0, 'u'); 
end; 

nonterm:  case  so. far  of 
0:  begin  share  w. value  (fir s.walk)\ 

walk(f(rs.xvalk,  selector  mod  share  ^firsJabels);  sofar*—l; 
end; 

1:  walk (seco. walk ^  selector  div  share ^  secoJabels); 
end; 
end; 

Tliis  code  is  used  iu  section  60. 
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C3.  Interacting  with  tlic  u^rr.  The  helj>  j)ro{T{liiro  prints  inforination  for  the  user 
by  roadnn’  frcnn  the  Me  lulp.txt  and  printirif:;  tlie  contents  on  the  tty.  Parameters  that  are 
easy  to  change  are  described  at  the  end  of  the  help  message. 

(Command  processing  63}  := 
procedure  help; 

begin  rr<<et  {help. file ,  'HELP  .TXT [1  ,DHG]  '); 

repeat  Teadln{help  .file  buffer  :  i.line.sizc);  WTiteln(tty  .Lhuffer  :  iJine.size); 
until  €of{hclp.fLle); 

ivritclu  (tty  .buffer. ,^ize  :  0,  'ucharactersuoiiuaulincuofuinput '); 
writcln((ty,  max. prod. symbols  :  0,  'usymbolsuinuauproductionupossibility '); 
writcln{tty ,  mai..cour}tableJerminal3  :  0, 

'ijsymbolsuthatuareucriticalutoutheucounting'); 
writeln(ity ,  rnaxJotal  : 

'utotciluoccurrencesuofuedchucriticalusymboluinutheuderivedustring'); 

end; 

See  also  section  64. 

This  code  is  nsctl  in  section  1. 
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64.  Command  Line  Processor. 

This  procedure  consists  of  three  nested  loops  at  three  different  levels  of  the  command 
line  processor.  The  outside  loop  prompts  with  >,  tUid  the  label  new^command  marks  the 
end  of  th.is  U)op.  The  inner  loops  appear  in  subsequent  modules. 

With  a  few  exceptions,  the  command  line  processor  ignores  all  but  the  first  letter  of  a 
line  of  input. 

define  quit  =  5 
define  new. command  =  6 
deHne  new.size  =  7 
define  new.selection  =  8 

(Command  processing  63)  +  = 
procedure  commands; 

label  quit,  new.size,  new.command,  new.selection; 
y^r  i,  column,  selector:  integer;  c:  char  ;  loc:  m.coord; 

limit ,  start. time,  heap. bottom:  integer;  labels  ,newJabel:  Lptr; 
begin  writeln(tty); 

writeln(tty ,  ' PI eas euus euC api tal s . uTheuHELPuCommanduprintSuins true t ions . '); 
mark  {heap. bottom); 
while  true  do 

begin  M;r*7c/n(f/T/);  write  {tty , 'CommQJid>u')\  i.line{true); 

case  i.bujfer[l]  of 

'H':  help  :  {  Help  command  } 

'G {  Cjrainnmr  command  } 

begin  for  c  'A'  to  'Z'  do  s.table[c]. status  ^  undefined; 
release  {heap.bottom);  loading. grammar;  done.already  ♦—nil; 
end; 

'S':  (Process  a  size  command  65); 

'U':  goto  ^ut7;  {  Up  command  } 

others:  writeln{tty ,  'Error  luunrecognizeducommand') 

end; 

new.command:  end; 
quit:  end; 

65.  Hero  is  the  middle  loop,  it  prompts  for  various  structure  sizes. 

(Process  a  size  command  C5)  = 

while  true  do 

begin  (Prompt  for  the  start  symbol  and  the  characteristic  vector  desired  66); 
start.time  runtime;  {  Tiic  user’s  reqiicst  is  in  loc  } 
m.locatr{loc):  writ€ln{tty,  'Thereu^reu^  count  {loc)  :  0,  'ystructures '); 
writcln{tly ,  'Runtime :u',  (rt/nhrnc  -  start.time)  :  0,  'umsecuOfuCPU'); 
if  count  {loc)  >  0  then  (Walk  through  selected  strings  until  the  user  types  UP  07); 
new.size:  end 
This  code  is  used  in  section  64. 
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66.  Tlio  user  sp(Tifios  a  starting  symbol  for  the  derivation,  and  gives  the  desired  distri- 
bution  of  characters  for  the  terniinal  string. 

(Prompt  for  the  start  symbol  and  the  characteristic  vector  desired  ce)  = 
wr{teln(tty)\  wrifc(tty,  'StartuNonterminal»u');  U{ne{tnie)\ 
if  {Lbnjferll}  =  'U')  A  (iMffer[2]  -  'P')  then  goto  nexv^command; 
if  s.tah!elt_buj[fcr\{]]. status  ^  nonterm  then 
begin  wrttcln[ity ^  Error :u',  'ushouldubeuaunonterminal '); 

goto  new,size\ 
end; 

loc, symbol  ^  i^hujfer[i]; 

WTiteln[tty,  'N^n^beruofuoccurrencesuof u(Limitu=u%  maxJotal  :  2,  ')  '); 
for  t  ^  1  to  number  .indices  do 

begin  write  (tty,  index.symbols[il  "'s»u');  read(tty,locAnt{i]), 
while  <  0)  V  >  xnax.total)  do 

begin  writ€(tty.  'Error  :ulimitu=u%  :  0,  '  .utryuagaiiiu.  .  -u'); 

read  (tty  Joe  .intli])\ 

end; 

end 

This  co<le  is  used  in  section  65. 


67.  Here  is  the  innermost  loop:  the  user  supplies  an  integer  identifying  a  particular 
stnicturo,  or  makes  a  RANDOM  reqnnst. 

(Walk  tlirongh  selected  strings  until  the  user  types  UP  67}  = 
begin  while  true  do 
begin  labels  nil;  limit 
for  1  ^  1  to  limit  do 

begin  n>:w(newJabel):  new. label]. lab  ^  limit  -  t  +  1;  new.label ] .Lnext  i-  labels- 

labels  new  .label] 

end; 

writ eln [tty)-,  writeln(tty.  'Whichuoneuwouldoyouulike?'); 
writeiUy.  'EnterufOu-  .un-l]uoruRANDOM»>u');  i.line (false); 
case  i.buffer[l]  of 

'U':  goto  new.stzc:  {Up  command} 

'R':  {  Random  command  } 

begin  tL>alk(m.acces.<)(loc),trunc(Tan(lom(i))  ♦  w.value(m.access(loc))),  labels); 
end;  ’ 

others:  (Select  a  particular  structure  68) 

end; 

new. selection:  end; 
end 

This  cotle  U  usc<l  in  section  65. 
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68.  The  user  hcis  requested  a  specific  structure  by  number. 

{ Select  a  particular  structure  68 )  = 
begin  {  Convert  the  contents  of  iJ)ujJer  to  a  number} 
column  1;  selector  0; 
while  column  <  Lline.stzc  do 

begin  if  -^(i^buffer [column]  E  ['0'  ..  '9'])  then 

begin  WTit€ln[tty ^  'Error runumeraluorucommanduexpected'); 

goto  new^selection] 

end; 

selector  ^  selector  ♦  10  +  ord{i, buffer  [column])  -  ord{'0');  column  ^  column  +  1; 
end; 

if  {selector  >  0)  A  {selector  <  count{loc))  then  walk {m^access{loc),  selector ,  labels) 

else  writeln{ttyy  'Error :uS elect ionuOUtuofuTange '); 

end 

This  code  is  usci!  in  section  67. 
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Sample  Execution 

The  preceding  program  executes  as  follows: 

Please  use  capitals.  The  HELP  coniinajid  prints  instructions, 

Cominand>  HELP 

Level  in  the  command  line  processor  is  indicated  by  the  number  of  >  signs 
following  a  prompt.  At  any  level,  the  UP  command  shifts  the  system  to  a 
higher  level,  A  summary  of  top  level  commands  follows: 

Print  this  file 
Exit  from  the  program 
Accept  a  grammar  from  the  user 

Accept  a  specification  from  user  for  the  size  of  objects  to 
be  counted,  selected,  or  generated  at  random. 

After  a  GRAMMAR  command,  the  program  asks  for  a  classification  of  input 
letters.  Letters  can  be  designated  as:  1)  The  special  labelled  character. 
2)  Terminals  that  are  not  labelled,  but  figure  in  the  specification  of  the 
problem  size.  3)  Terminals  that  will  be  ignored  in  the  counting,  but 
printed  in  the  final  result,  4)  Nonterminals.  At  each  prompt  the  user 
supplies  one  or  more  letters,  separated,  if  desired,  by  spaces  or  commas. 

E  is  the  built  in  empty  string.  It  should  not  be  redefined. 

Following  the  declaration  of  letters,  the  program  will  ask  for 
productions.  It  expects  a  vertical  bar  to  separate  production 
possibilities,  and  an  up  arrow  to  indicate  a  box  superscript.  The  UP 
command,  when  issued  at  the  beginning  of  a  line  of  input,  returns  the 
program  to  its  top  level. 

In  response  to  a  SIZE  command,  the  program  asks  the  user  for  a  start 
nonterminal  and  then  the  number  of  occurrences  of  the  special  character  and 
each  of  the  terminals  declared  to  be  important  to  the  specification  of 
problem  size.  It  reports  the  results  of  the  counting  and  then  asks  the 
user  to  select  a  particular  object.  The  user  can  supply  an  integer  or  use 
the  RANDOM  command  for  less  predictable  results. 

Here  is  a  sample  program  execution  for  labelled  trees: 


>HELP 

>UP 

>GRAMMAR 

>SIZE 


Please  use  capitals.  The  HELP  command  prints  instructions. 

Command>  GRAMMAR 

Labelled  Character»  X 
Counted  Terminal (8)» 

Uncounted  Terminal  (s)»  L,  H 
Nonterminal  (8)»  ST 
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Production»  T-^XS 

Loaded:  T  X  S  with  boxed  position  0 
Production»  S  L  T"  H  S  I  E 
Loaded:  S  L  T  H  S  with  boxed  position  2 

Loaded:  S  E  with  boxed  position  0 

Production»  UP 

The  following  nonterminals  can  derive  pseudo  empty  strings:  S 
The  safe  order  is:  T  S 

Coinmand>  SIZE 

Start  Nonterminal»  T 

Number  of  occurrences  of  (Limit  =  12) 

X's»  3 

There  are  9  structures 
Runtime:  3  msec  of  CPU 

Which  one  would  you  like? 

Enter  [0  . .  n-1]  or  RAND0M»>  0 

XI  L  X2  H  L  X3  H 

Which  one  would  you  like? 

Enter  [0  . .  n-1]  or  RAND0M»>  1 

X2  L  XI  H  L  X3  H 

Which  one  would  you  like? 

Enter  [0  . .  n-1]  or  RAND0M»>  RANDOM 

X2  L  XI  H  L  X3  H 

Which  one  would  you  like? 

Enter  [0  ..  n-1]  or  RAND0M»>  UP 

Start  Nonterminal»  T 

Number  of  occurrences  of  (Limit  =  12) 

X’8»  10 

There  are  1000000000  structures 
Runtime:  12  msec  of  CPU 

Which  one  would  you  like? 

Enter  [0  . .  n-1]  or  RAND0M»>  R 

II  L  J3  L  XQ  L  X8  L  X4  L  X7  L  X5  L  XIO  L  X2  H  H  H  H  H  L  X6  H  H  H  H 
Which  one  would  you  like? 

Enter  [0  ..  n-1]  or  RAND0M»>  UP 

Start  Nonterminal»  UP 

Command>  UP 


Exit 
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The  following  limits  are  built  into  the  program: 

1  special  labelled  character  in  a  production  possibility, 

2  nonterminals  in  a  production  possibility. 

The  follo'^'ing  limits  can  be  modified  by  recompilation: 

140  characters  on  a  line  of  input 
7  symbols  in  a  production  possibility 

3  symbols  that  are  critical  to  the  counting 

12  total  occurrences  of  each  critical  symbol  in  the  derived  string 
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Tho  next  example  uses  a  grammar  to  encode  Fibonacci  sequences.  These  are  strings 
of  +  (PS)  and  -  (MS)  signs  having  no  consecutive  minus  signs. 

Coimnand>  GRAl-iMAR 

Labelled  Character» 

Enter  a  single  labelled  character  (dummy  if  necessary)  . . .  Z 
Counted  Terminal (s)»  S 
Uncoimted  Terminal (8)»  P  M 
Nonterminal (8)»  A  B 

Production»  A  PS  A  I  MS  B  |  E 

Loaded:  A  P  S  A  with  boxed  position  0 

Loaded:  A  M  S  B  with  boxed  position  0 

Loaded:  A  E  with  boxed  position  0 
Production»  B  PS  A  I  E 

Loaded:  B  P  S  A  with  boxed  position  0 

Loaded:  B  E  with  boxed  position  0 
Production»  UP 

The  following  nonterminals  can  derive  pseudo  empty  strings:  A  B 
The  safe  order  is:  A  B 

Command>  SIZE 

Start  Nonterminal»  A 

Nximber  of  occurrences  of  (Limit  =  12) 

Z’s»  0 
S^8»  2 

There  are  3  structures 
Runtime:  3  msec  of  CPU 


Which 

one 

would  you 

like? 

Enter 

[0 

. .  n-l3  or 

RANDCM»> 

0 

PSP 

S 

Which 

one 

would  you 

like? 

Enter 

CO  . 

. .  n-1]  or 

RAND0M»> 

1 

P  S  M 

S 

Which 

one 

would  you 

like? 

Enter 

[0  . 

. .  n-1]  or 

RANOOM»> 

2 

MSP 

S 

Which 

one 

would  you 

like? 

Enter 

[0  . 

. .  n-1]  or 

RAND0M»> 

UP 

Start 

Nont€rminal» 

A 

Number  of  occurrences  of  (Limit  «  12) 
Z*8»  0 
S'e»  3 


136 


A  CENERAL-runrOSE  GENERATOR  OF  COMBINATORIAL  OBJECTS 


There  are  6  structures 
Runtime:  1  msec  of  CPU 

V/hich  one  would  you  like? 

Enter  [0  ..  n**!]  or  RAND0M»>  UP 

Start  Nonterminal»  A 

Number  of  occurrences  of  (Limit  «  12) 

Z's»  0 

S*8»  4 

There  are  8  structures 
Runtime:  2  msec  of  CPU 

Which  one  would  you  like? 

Enter  [0  .  .  n-1]  or  RAND0M»>  UP 


Steart  Nonterminal»  UP 
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This  last  grammar  counts  unordcrcd,  labelled  trees  according  to  their  leaves  and  single 
descendant  nodes.  Loaves  arc  marked  with  an  A,  and  single  dcsccndents  arc  marked  with 
a  B. 

Command?*  GRAMMAR 

Labelled  Character»  X 
Counted  Terminal(8)»  A  B 
Uncounted  Terminal(s)»  L  H 
Nonterminal (8)»  STM 

Production»  T  ♦  X  S 
Loaded:  T  X  S  with  boxed  position  0 
Production»  S^AlBLTHlLT^HM 
Loaded:  S  -►  A  with  boxed  position  0 
Loaded:  S  B  L  T  H  with  boxed  position  0 

Loaded:  S  L  T  H  M  with  boxed  position  2 

Production»  M->LTH|LT^HM 
Loaded:  M  L  T  H  with  boxed  position  0 

Loaded:  M  L  T  H  M  with  boxed  position  2 

Production»  UP 

The  following  nonterminals  can  derive  pseudo  empty  strings: 

The  safe  order  is:  STM 

Command>  SIZE 

Start  Nonterminal»  3 

Error:  capital  letterCs)  expected,  try  again  T 
Number  of  occurrences  of  (Limit  *  12) 

X’8»  3 
A*fe»  2 
B*s»  0 

There  are  3  structures 
Runtime:  56  msec  of  CPU 

Which  one  would  you  like? 

Enter  [0  . .  n-1]  of  RAND0M»>  0 
XI  L  X2  A  H  L  X3  A  H 
Which  one  would  you  like? 

Enter  [0  n-1]  or  RAND0M»>  1 

X2LX1AHLX3AH 

Which  one  would  you  like? 

Enter  [0  ..  n-1]  or  RAND0M»>  2 
X3  L  XI  A  H  L  X2  A  H 
Which  one  would  you  like? 

Enter  [0  n-1]  or  RAND0M»>  UP 


138  A  GEKEnAL-PUrj’OSE  GENERATOR  OF  COMBINATORIAL  OBJECTS 

Start  Nontcrminal»  T 

Nur.ber  of  occurrences  of  (Limit  *  12) 

X's»  8 
A’8»  4 
B'8»  3 

There  are  58800  structxires 
Runtime:  2382  msec  of  CPU 

Which  one  would  you  like? 

Enter  [0  n-l]  or  RAND0M»>  RANDOM 

X8BLX7LX3BLX1BLX6AHHHLX2AHLX4AHLX6AHH 
Which  one  would  you  like? 

Enter  [0  ..  n-1]  or  RAND0M»>  UP 

Start  Nonterminal»  UP 


Command>  UP 
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1.  Enumeration  under  the  Cyclic  Group.  This  program  reads  a  number  n  from 
the  user’s  terminal,  then  outputs  all  sequences  of  zeros  and  ones  of  length  n  that  arc  distinct 
under  the  cyclic  group.  Each  vector  output  is  lexicographically  largest  in  its  equivalence 
class,  find  the  vectors  are  output  in  decreasing  lexicographic  order.  For  example,  when 
n  =  4,  the  outputs  arc  1111,  1110,  1100,  1010,  1000,  0000.  The  program  also  reports 
the  number  of  distinct  elements  found,  which  can  be  checked  against  Polya’s  enumeration 
formula: 

i\n 

Here  ^(t)  is  Euler’s  phi  function;  it  counts  the  number  of  integers  less  than  i  that  are 
relatively  prime  to  i. 

This  program  is  efficient  in  that  it  doc.sn’t  scan  the  entire  set  of  2^  vectors  of  zeros 
and  ones.  Instead,  the  program  uses  an  “incorrect”  block  move  instruction  to  skip  over 
large  pieces  of  the  set.  One  nice  consequence  of  this  organization  is  that  the  test  to  reject 
unwanted  vectors  is  relatively  simple  (see  module  “Test  and  output  if  good”  below). 

define  max^size  =  30 
define  quit  =  10 

program  cyclic  {tty  ^  output); 
label  quit; 

type  smalLinteger  =  0  . .  max.size; 
var  (Variables  used  by  the  program  2) 

begin  while  /r«c  do  {  The  program  will  solicit  numerous  problem  sizes  } 
begin  WTitc(ttyy  'PleaseuEnteruProblemuSizeuu');  read  {tty  ^n); 

(Do  it  for  n  3); 

quit:  writeln{tty,  'TotaluNumberuOfuVector8uFoundu“u%  :  3, 

'LJu<ctr>Cu^.f  inishutouseeufile'); 
writeln {output ,  'Nu*',n  :  4,  ■  ■  X^ount. count  :  3); 

end; 


AN  I' XAMI'M:  or  rOLYA-nCDrilXD  ICNUMI-nATION 


1 10 


2.  1  ho  only  data  structure  is  an  array 

array  of  0/1. 


rector  of  hooleans  wliich  the  user  perceives  as  an 


(Variables  )ise<l  by  the  proj^rarn  2)  ^ 

n,  siriiill _i7}fr{jrr \  |  ri,  the  problem  size,  is  read  from  the  tormina!  } 

t:  .-iTnalLintrijrr-,  {  i  is  a  .scratch  variable,  ii.scd  for  it  ation.s  through  verAor  } 

vector :  .irray  of  hookan ;  {  tiie  boolcaos  will  be  printed  as  zeros  or  ones  } 

reset:  small  integer:  {  the  location  of  the  rijthtmost  true  in  vector  } 

next. to. ropy:  smallJnteger;  { the  next  boolean  in  vector  to  be  copied  } 

count  :  integer  :  {  cotnits  number  of  ineqiiivalent  vectors  found  } 

This  rode  is  used  in  section  1. 


3.  The  central  loop  of  the  program  scans  the  array  for  the  rightmost  one,  resets  this  one 
to  zero,  and  then  copies  the  beginning  of  the  array  into  the  positions  following  the  reset 
one.  This  operation  is  likely  to  generate  an  isomorphically  distinct  vector  of  zeros  and  ones. 
( Do  it  for  n  3 )  = 

count  0; 

for  I  ^  0  to  n  do  vector  [i]  true; 

(Print  the  array  4); 

vector  [ri]  fake  ;  {  Hereafter  the  last  position  in  vector  is  fixed  at  zero  } 

( Print  the  array  4 ); 
while  tr7ie  do 

begin  (Find  the  rightmost  one  and  reset  it;  goto  quit  if  all  zeros  5); 

(Copy  from  the  beginning  of  the  vector  to  the  zeros  after  reset  6); 

(Test  and  output  if  good  7); 
end; 

This  code  is  used  in  section  1. 

4.  Convert  the  boolean  array  vector  to  0/1  and  print  the  array  in  output. 

( Print  the  array  4 )  = 

begin  for  t  4-  1  to  n  do 
if  vector [ij  then  write{*l') 
else  wTite(*0')\ 
count  count  +1;  writeln; 
end; 

This  code  is  use<l  in  sections  3»  3,  and  7. 

5.  The  rightmost  one  in  vector  is  set  to  The  variable  reset  points  to  the  clianged 

location.  When  vector\l  ..  nj  is  all  zero,  the  otherwise  useless  one  entry  at  vectorlO]  forces 
an  exit.  *  ' 

(Find  the  riglitiiiost  one  and  reset  it;  goto  quit  if  all  zeros  5)  = 
reset  4—  n; 

repeat  reset  4—  reset  —  1; 
until  vector  [reset]; 
if  reset  =  0  then  goto  quit; 
vector  [reset]  4—  false; 

This  code  is  u.^^rd  in  section  3, 
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6,  This  is  a  block  move  instruction  with  origin  1  and  destination  reset  +  1.  It  works 
‘‘incorrectly”  in  the  sense  that  the  origin  area  may  o’^erlap  the  destination  area;  since  the 
direction  of  copying  is  forward,  one  entry  may  be  replicated  several  times.  This  is  precisely 
the  behavior  desired  for  the  algorithm. 

(Copy  from  the  beginning  of  the  vector  to  the  zeros  after  reset  6)  = 
nextJo^copy  +-  1;  i  +-  reset  -r  1; 
while  t  7^  n  do 

begin  vector  [i]  ^  vector  [next Jo^copy];  i  i  +  1;  nextJo^copy  nextJo.copy  -f  1; 
end; 

This  code  is  used  in  section  3. 

The  newly  created  vector  is  “good”  if  the  next  item  in  line  for  copying  is  one.  This 
item  would  normally  land  in  the  last  entry  of  vector,  which  is  permanently  fixed  at  zero,  so 
we  know  that  the  first  part  of  vector  is  lexicographically  larger  than  the  portion  of  vector 
following  reset. 

In  special  cases  we  allow  the  next  item  in  line  for  copying  to  be  zero.  This  corresponds 
to  non-prime  n  where  there  is  a  repeated  pattern  in  the  array.  The  mod  below  checks  for 
this  posibility. 

( Tost  and  output  if  good  7 )  = 
if  vector  [next  Jo.copy]  V  (n  mod  reset  =  0)  then 
begin  (Print  the  cirray  4) 
end; 

This  code  is  used  in  section  3. 
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